Self-Distilled Disentangled Learning for Counterfactual Prediction

TL;DR

The proposed SD2 framework ensures theoretically sound independent disentangled representations without intricate mutual information estimator designs for high-dimensional representations and confirms the effectiveness of the approach in facilitating counterfactual inference in the presence of both observed and unobserved confounders.

Abstract

The advancements in disentangled representation learning significantly enhance the accuracy of counterfactual predictions by granting precise control over instrumental variables, confounders, and adjustable variables. An appealing method for achieving the independent separation of these factors is mutual information minimization, a task that presents challenges in numerous machine learning scenarios, especially within high-dimensional spaces. To circumvent this challenge, we propose the Self-Distilled Disentanglement framework, referred to as $SD^2$. Grounded in information theory, it ensures theoretically sound independent disentangled representations without intricate mutual information estimator designs for high-dimensional representations. Our comprehensive experiments, conducted on both synthetic and real-world datasets, confirms the effectiveness of our approach in facilitating counterfactual inference in the presence of both observed and unobserved confounders.

Figures (7)

• Figure 1: General causal structure. The underlying confounders $C$ in observed pre-treatment features $X$ and unobserved confounders $U$ result in spurious relations rather than causal relations between treatment $T$ and outcome $Y$. We aim to disentangle mutually independent representations of $Z$, $C$, and $A$ from $X$ without the design of intrigue mutual information estimators.
• Figure 2: A motivating Venn diagram of mutual information between $R_c$, $R_a$ and $Y$ during training phase.
• Figure 3: Self-distillation unit for minimizing $\mathcal{L}_{c}^{z}$.
• Figure 4: Experimental results under continuous scenario on Demand-0-1. Among all methods, $VSD^2$ achieves the best and most stable results.
• Figure 5: Radar charts that visualize the capability of $VSD^2$ and a classical causal disentangled learning baseline DRCFR. Every vertex on the polygons represents a synthetic dataset with setting $mv$-$mz$-$mc$-$ma$-$mu$. The red and blue denote the contribution of true variables and other variables to the decomposed representations. The results demonstrate our method achieves much better identification performance of all three underlying factors in all synthetic datasets compared with DRCFR.

Theorems & Definitions (3)

• definition 1
• theorem 1
• corollary 1