Adversary-Free Counterfactual Prediction via Information-Regularized Representations
Shiqin Tang, Rong Feng, Shuxin Zhuang, Hongzong Li, Youzhi Zhang
TL;DR
This work addresses bias in observational causal inference by framing counterfactual prediction as an information-regularized representation learning problem. It avoids adversarial training by directly penalizing mutual information between the learned representation $Z$ and treatment $T$, using a tractable variational bound to couple outcome prediction with a treatment-agnostic constraint. The static (SICE) and dynamic (DICE) formulations yield competitive or superior performance across synthetic and real data, particularly as treatment dimensionality grows. The approach offers stable training, principled generalization properties, and practical applicability to high-dimensional, multi-treatment settings such as clinical decision support and sequential treatment policies.
Abstract
We study counterfactual prediction under assignment bias and propose a mathematically grounded, information-theoretic approach that removes treatment-covariate dependence without adversarial training. Starting from a bound that links the counterfactual-factual risk gap to mutual information, we learn a stochastic representation Z that is predictive of outcomes while minimizing I(Z; T). We derive a tractable variational objective that upper-bounds the information term and couples it with a supervised decoder, yielding a stable, provably motivated training criterion. The framework extends naturally to dynamic settings by applying the information penalty to sequential representations at each decision time. We evaluate the method on controlled numerical simulations and a real-world clinical dataset, comparing against recent state-of-the-art balancing, reweighting, and adversarial baselines. Across metrics of likelihood, counterfactual error, and policy evaluation, our approach performs favorably while avoiding the training instabilities and tuning burden of adversarial schemes.