Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conformal Inference for Counterfactuals and Individual Treatment Effects with Experiment Attrition

Xiangyu Song

Abstract

Attrition in survey and field experiments presents a challenge for social science research. Common approaches to deal with this problem -- such as complete case analysis, multiple imputation, and weighting methods -- rely on strong assumptions that may not hold in practice. This paper introduces a new method that combines recent advances in statistical inference with established tools for handling missing data. The approach produces prediction intervals for treatment effects that are both robust and precise. Evidence from simulation studies shows that the method achieves better coverage and produces narrower intervals than common alternatives. The reanalysis of two recently published experiment studies illustrates how this framework allows researchers to compare treatment effects across participants who remain in the study, those who drop out, and the full sample. Taken together, these results highlight how the proposed approach provides a stronger foundation for causal inference in the presence of attrition.

Conformal Inference for Counterfactuals and Individual Treatment Effects with Experiment Attrition

Abstract

Attrition in survey and field experiments presents a challenge for social science research. Common approaches to deal with this problem -- such as complete case analysis, multiple imputation, and weighting methods -- rely on strong assumptions that may not hold in practice. This paper introduces a new method that combines recent advances in statistical inference with established tools for handling missing data. The approach produces prediction intervals for treatment effects that are both robust and precise. Evidence from simulation studies shows that the method achieves better coverage and produces narrower intervals than common alternatives. The reanalysis of two recently published experiment studies illustrates how this framework allows researchers to compare treatment effects across participants who remain in the study, those who drop out, and the full sample. Taken together, these results highlight how the proposed approach provides a stronger foundation for causal inference in the presence of attrition.

Paper Structure

This paper contains 44 sections, 9 theorems, 118 equations, 21 figures, 5 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Current Approaches to Deal with Attrition in Experimental Settings
  3. Problem Setup and Notation
  4. Conformal Inference with Covariate Shift
  5. Conformal Inference
  6. Covariate Shift
  7. Weighted Conformal Inference
  8. Conformal Inference with Attrition
  9. Conformal Inference with Semiparametric Efficient Estimator
  10. Conformal Inference for Counterfactuals and ITE on Observed Group
  11. Conformal Inference for ITE with Attrition
  12. Estimation Algorithm
  13. Asymptotic Coverage of the Prediction Interval for ITE with Attrition
  14. Simulation Studies
  15. Empirical Application
  16. ...and 29 more sections

Key Result

Lemma 1

Under Assumption asm1, we have $\blacktriangleleft$$\blacktriangleleft$

Figures (21)

  • Figure 1: Workflow of Algorithm
  • Figure 2: MC Simulation Results of Conformal Inference for ITE with Attrition
  • Figure 3: MC Simulation Results of Conformal Inference for ITE with Attrition
  • Figure 4: Comparison of Empirical Coverage of Prediction Intervals for ITE with Attrition
  • Figure 5: Comparison of Average Length of Prediction Intervals for ITE with Attrition
  • ...and 16 more figures

Theorems & Definitions (22)

  • Lemma 1: Setting 1 in Theorem 1 of gao2025role
  • proof
  • Theorem 1: Lemma 1 of yang2024doubly
  • proof
  • Lemma 2
  • proof
  • Theorem 2
  • proof
  • Theorem A.1
  • Definition A.1: Exchangeability
  • ...and 12 more