Denoising the US Census: Succinct Block Hierarchical Regression

Abstract

The US Census Bureau Disclosure Avoidance System (DAS) balances confidentiality and utility requirements for the decennial US Census (Abowd et al., 2022). The DAS was used in the 2020 Census to produce demographic datasets critically used for legislative apportionment and redistricting, federal and state funding allocation, municipal and infrastructure planning, and scientific research. At the heart of DAS is TopDown, a heuristic post-processing method that combines billions of private noisy measurements across six geographic levels in order to produce new estimates that are consistent, more accurate, and satisfy certain structural constraints on the data. In this work, we introduce BlueDown, a new post-processing method that produces more accurate, consistent estimates while satisfying the same privacy guarantees and structural constraints. We obtain especially large accuracy improvements for aggregates at the county and tract levels on evaluation metrics proposed by the US Census Bureau. From a technical perspective, we develop a new algorithm for generalized least-squares regression that leverages the hierarchical structure of the measurements and that is statistically optimal among linear unbiased estimators. This reduces the computational dependence on the number of geographic regions measured from matrix multiplication time, which would be infeasible for census-scale data, to linear time. We incorporate the additional structural constraints by combining this regression algorithm with an optimization routine that extends TDA to support correlated measurements. We further improve the efficiency of our algorithm using succinct linear-algebraic operations that exploit symmetries in the structure of the measurements and constraints. We believe our hierarchical regression and succinct operations to be of independent interest.

Figures (10)

• Figure 1: The flow of data through the US Census Disclosure Avoidance System.
• Figure 2: Illustration of the geocode tree in the US Census.
• Figure 3: Average accuracy improvements for our algorithm for queries aggregating by state, county, tract, and block group, and error distributions of queries across each geographic unit at the county and tract levels.
• Figure 4: Average accuracy improvements for race and group quarters type queries. For race categories, AIAN=American Indian and Alaska Native, NHPI=Native Hawaiian and Pacific Islander, SOR=Some Other Race. For group quarters categories, HH=Household, CF=Correctional facilities for adults, JF=Juvenile facilities, NF=Nursing facilities/Skilled-nursing facilities, OI=Other institutional facilities, SH=College/University student housing, MQ=Military quarters, ON=Other noninstitutional facilities.
• Figure 5: Mean error (bias) for total population of each population bin at county and tract levels, for each replicate run. Population bins are mutually exclusive.

Theorems & Definitions (26)

• Definition 5.1: Matrix Pseudoinverse
• Definition 5.3: Discrete Gaussian Distribution; see, e.g., canonne20discrete
• Definition 5.4: BLUE
• Lemma 5.5: Gauss--Markov Theorem
• Lemma 5.6
• Lemma 5.7: BLUE for $\mathsf{ECGLR}_{W,\Sigma}^{R,r}$ with $\Sigma=I$ (e.g., amemiya1985advanced)
• proof
• proof : Proof of \ref{['lem:ecgls']}
• Lemma 5.8: e.g., Section 5.5.3 in boyd2014convex
• Proposition 5.9