Multi-CATE: Multi-Accurate Conditional Average Treatment Effect Estimation Robust to Unknown Covariate Shifts

TL;DR

This work uses methodology for learning multi-accurate predictors to post-process CATE T-learners to become robust to unknown covariate shifts at the time of deployment to establish a connection between methods developed for multi-distribution learning and achieve appealing desiderata in causal inference and machine learning.

Abstract

Estimating heterogeneous treatment effects is important to tailor treatments to those individuals who would most likely benefit. However, conditional average treatment effect predictors may often be trained on one population but possibly deployed on different, possibly unknown populations. We use methodology for learning multi-accurate predictors to post-process CATE T-learners (differenced regressions) to become robust to unknown covariate shifts at the time of deployment. The method works in general for pseudo-outcome regression, such as the DR-learner. We show how this approach can combine (large) confounded observational and (smaller) randomized datasets by learning a confounded predictor from the observational dataset, and auditing for multi-accuracy on the randomized controlled trial. We show improvements in bias and mean squared error in simulations with increasingly larger covariate shift, and on a semi-synthetic case study of a parallel large observational study and smaller randomized controlled experiment. Overall, we establish a connection between methods developed for multi-distribution learning and achieve appealing desiderata (e.g. external validity) in causal inference and machine learning.

Figures (19)

• Figure 1: Schematic of setting 1 (external shift), and setting 2 (learning from large observational and small RCT data). We propose multi-accuracy (MC-Boost) auditing as an "off-the-shelf" procedure to improve the downstream robustness of CATE learners to unknown covariate shifts in both settings.
• Figure 2: Average MSE of CATE estimation by shift intensity and training set size for post-processed (multi-calibrated) T- and DR-learner and benchmark methods in simulation studies (external shift setting).
• Figure 3: Average MSE of CATE estimation by shift intensity and training set size for post-processed (multi-calibrated) T- and DR-learner and benchmark methods in simulation studies (observational data with RCT setting).
• Figure 4: Average absolute bias and MSE by clinical trial sample size in WHI data application
• Figure 5: Bias of ATE estimation by shift intensity and training set size for different CATE estimation methods (\ref{['para:simu1a']}). The distribution of bias scores over simulation runs is shown. Given an external shift between training and test data, DR-learner-MC-Ridge and T-learner-MC-Ridge perform best among the shift-blind methods that had no access to the shifted target distribution.

Theorems & Definitions (9)

• Definition 1: Multi-accuracy
• Definition 2: Multiaccuracy auditing
• Remark 1: Relation to conditional moment restrictions
• Proposition 1: Multi-accuracy implies robust estimation of the ATE via regression adjustment
• Corollary 1
• Proposition 2
• Proposition 3
• proof
• proof : Proof of \ref{['prop-uaisdr']}