Advancing Counterfactual Inference through Nonlinear Quantile Regression

TL;DR

This work reframes counterfactual inference, which traditionally relies on a predefined structural causal model, as an extended nonlinear quantile regression problem. It proves that, under mild monotonicity assumptions on the unobserved noise transformation, the counterfactual outcome $Y_{X=x'}|X=x,Y=y,Z=z$ equals the $τ$-th quantile of $P(Y|X=x',Z=z)$ where $Y=y$ is the $τ$-th quantile of $P(Y|X=x,Z=z)$, enabling identifiability without recovering the true SCM or noise. The authors then develop a practical neural-network implementation using a bi-level optimization scheme to estimate the target quantile and its regression function, with a data-dependent upper level that allows efficient predictions for unseen samples and a generalization bound based on Rademacher complexity. Empirical results across synthetic, tabular, and image datasets demonstrate superior statistical efficiency and competitive generalization performance relative to state-of-the-art counterfactual methods, validating the theoretical claims and highlighting robustness to various causal models. This framework broadens reliable counterfactual analysis to settings with unknown SCMs and minimal distributional assumptions, with potential impact on causal inference in diverse applied domains.

Abstract

The capacity to address counterfactual "what if" inquiries is crucial for understanding and making use of causal influences. Traditional counterfactual inference, under Pearls' counterfactual framework, typically depends on having access to or estimating a structural causal model. Yet, in practice, this causal model is often unknown and might be challenging to identify. Hence, this paper aims to perform reliable counterfactual inference based solely on observational data and the (learned) qualitative causal structure, without necessitating a predefined causal model or even direct estimations of conditional distributions. To this end, we establish a novel connection between counterfactual inference and quantile regression and show that counterfactual inference can be reframed as an extended quantile regression problem. Building on this insight, we propose a practical framework for efficient and effective counterfactual inference implemented with neural networks under a bi-level optimization scheme. The proposed approach enhances the capacity to generalize estimated counterfactual outcomes to unseen data, thereby providing an upper bound on the generalization error. Furthermore, empirical evidence demonstrates its superior statistical efficiency in comparison to existing methods. Empirical results conducted on multiple datasets offer compelling support for our theoretical assertions.

Figures (12)

• Figure 1: Illustration of our proposed quantile-based counterfactual estimation ($Z$ is omitted for illustration purpose). For a sample of interest ($X=x,Z=z,Y=y$), we estimate the quantile $\tau=P(Y\leq y|X=x,Z=z)=0.70$ with factual observations. Then the counterfactual outcome $Y_{X=x^\prime}$ is equal to the value $y^\prime$ which satisfy $P(Y\leq y^\prime|X=x^\prime, Z=z)=\tau$.
• Figure 2: Comparisons of the counterfactual predictions when traversing $X$ under two different forms of $g(E)$. Our method, based on Theorem \ref{['Theorem']}, demonstrates resilience to various forms of $g(E)$.
• Figure 2: The RMSE performance for counterfactual inference on image transformation datasets.
• Figure 3: One training step of our proposed bi-level implementation.
• Figure 4: Toy examples for counterfactual estimations. For the interested sample, we traverse the value of $X$ with $x^\prime$ and compare against with the true counterfactual $Y_{x=x^\prime}$. Our method is able to recover the true trajectory from factual observations under the five causal models.

Theorems & Definitions (8)

• Definition 1: Counterfactual outcomes Pearl2000
• Theorem 1
• Definition 2: Rademacher complexity bartlett2002rademacher
• Theorem 2
• proof : Proof
• Theorem 3
• Theorem 4
• Theorem 5