Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Causal Inference with Corrupted Data: Measurement Error, Missing Values, Discretization, and Differential Privacy

Anish Agarwal, Rahul Singh

TL;DR

A semiparametric model of causal inference with high dimensional corrupted data is formulated and a procedure for data cleaning, estimation, and inference with data cleaning-adjusted confidence intervals is proposed.

Abstract

The US Census Bureau will deliberately corrupt data sets derived from the 2020 US Census, enhancing the privacy of respondents while potentially reducing the precision of economic analysis. To investigate whether this trade-off is inevitable, we formulate a semiparametric model of causal inference with high dimensional corrupted data. We propose a procedure for data cleaning, estimation, and inference with data cleaning-adjusted confidence intervals. We prove consistency and Gaussian approximation by finite sample arguments, with a rate of $n^{ 1/2}$ for semiparametric estimands that degrades gracefully for nonparametric estimands. Our key assumption is that the true covariates are approximately low rank, which we interpret as approximate repeated measurements and empirically validate. Our analysis provides nonasymptotic theoretical contributions to matrix completion, statistical learning, and semiparametric statistics. Calibrated simulations verify the coverage of our data cleaning adjusted confidence intervals and demonstrate the relevance of our results for Census-derived data.

Causal Inference with Corrupted Data: Measurement Error, Missing Values, Discretization, and Differential Privacy

TL;DR

A semiparametric model of causal inference with high dimensional corrupted data is formulated and a procedure for data cleaning, estimation, and inference with data cleaning-adjusted confidence intervals is proposed.

Abstract

The US Census Bureau will deliberately corrupt data sets derived from the 2020 US Census, enhancing the privacy of respondents while potentially reducing the precision of economic analysis. To investigate whether this trade-off is inevitable, we formulate a semiparametric model of causal inference with high dimensional corrupted data. We propose a procedure for data cleaning, estimation, and inference with data cleaning-adjusted confidence intervals. We prove consistency and Gaussian approximation by finite sample arguments, with a rate of for semiparametric estimands that degrades gracefully for nonparametric estimands. Our key assumption is that the true covariates are approximately low rank, which we interpret as approximate repeated measurements and empirically validate. Our analysis provides nonasymptotic theoretical contributions to matrix completion, statistical learning, and semiparametric statistics. Calibrated simulations verify the coverage of our data cleaning adjusted confidence intervals and demonstrate the relevance of our results for Census-derived data.

Paper Structure

This paper contains 21 sections, 74 theorems, 70 equations, 6 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Model overview
  4. Data cleaning-adjusted confidence interval
  5. Finite sample analysis
  6. Case study: Effect of import competition using Census data
  7. Conclusion
  8. Appendix
  9. Data cleaning
  10. Error-in-variable regression
  11. Error-in-variable balancing weight
  12. Data cleaning-adjusted confidence intervals
  13. Additional examples
  14. Nonlinearity
  15. Data cleaning supporting details
  16. ...and 6 more sections

Key Result

Proposition 4.1

For $i\in\textsc{test}$, The alternative procedure of filling missing values with averages from train, denoted by $\bar{Z}_j^{\textsc{train}}$, gives

Figures (6)

  • Figure 1: Key assumption in Census data
  • Figure 2: Key assumption in simulated data
  • Figure 3: First pass --- OLS
  • Figure 4: Our approach adapts to the type and level of corruption.
  • Figure 5: Synthetic corruption
  • ...and 1 more figures

Theorems & Definitions (150)

  • Proposition 4.1: Filling with zeros is unbiased and simple
  • Proposition 4.2: Implicit data cleaning preserves independence
  • Proposition 4.3: The balancing weight exactly balances covariates
  • Remark 5.1
  • Theorem 5.1: Finite sample data cleaning rate
  • Remark 5.2
  • Remark 5.3
  • Theorem 5.2: Finite sample error-in-variable regression rate
  • Corollary 5.1: Simplified regression rate
  • Remark 5.4
  • ...and 140 more