Developing a 21st Century Census Curated Data Enterprise: A Bold New Scientific Approach for Official Statistics

Questions we now seek to answer often go beyond what the census or surveys alone can hope to address. A prominent example is the data equity problem. Limited sample sizes mean many official statistics lack demographic details or fail to represent underserved populations accurately. Small population groups and sparsely populated rural communities face accuracy issues due to small sample sizes and large margins of error. Privacy concerns further limit access to disaggregated data, preventing communities from fully understanding their challenges and solutions. This lack of data, or inaccurate data when it exists, makes identifying, measuring, and advancing equity challenging. Similar issues arise across domains, prompting researchers and policymakers to seek novel data sources -from structured numeric data in traditional databases to unstructured text scraped from the internet- to supplement traditional data, such as censuses and surveys to fill the data gaps and improve the quality and usefulness of statistical products. However, these efforts are often ad hoc, siloed, and duplicative, resulting in sub-optimal solutions.

That is the impetus for developing the 21st Century Curated Data Enterprise (CDE), an innovation in data science designed to create statistical products from diverse data types and build the infrastructure to support them. The Curated Data Enterprise, as the name implies, features an end-to-end curation model that captures the entire statistical product development process. Motivated by the Census Bureau’s needs, this initiative supports Census Bureau scientists and their data users to shift from a focus on single survey-driven data elements to a framework that integrates and curates multiple data sources. This integration aims to produce better social and economic measures, enhancing social science measurement and research.

This report describes and summarizes our research to address two tasks requested by the US Census Bureau:
Task 1: Identifying and Characterizing Stakeholder Communities and
Task 2: Developing a Research Agenda.

Just like putting humans on the moon, the Census Bureau has set clear objectives, is dedicating resources to meet the objectives, and is pursuing internal and external collaborative efforts to socialize and produce statistical products in a bold new way. This new approach aims to produce better measures of people, places, and economies, ultimately enhancing social science measurement and research.

Shipp S, Zhu Z, Jost S, Naymark J, Prewitt K, Salvo J. Thompson J, Snipp M, Becker-Medina E. (2024). Developing a 21st Century Census Curated Data Enterprise: A Bold New Scientific Approach for Official Statistics. University of Virginia. DOI: https://doi.org/10.18130/sg02-cy43