Causal Flow-based Variational Auto-Encoder for Disentangled Causal Representation Learning

TL;DR

DCVAE tackles causal disentanglement under interdependent generative factors by introducing causal flows into a supervised VAE framework. It uses a learnable conditional prior and a supervision term to align latent factors with ground-truth causes, enabling interventions and counterfactual generation. The approach demonstrates improved causal disentanglement and downstream task performance on Pendulum and CelebA datasets, outperforming state-of-the-art baselines. By integrating causal structure through flows rather than relying on fixed SCM priors, DCVAE offers a flexible, scalable pathway to uncover true causal relationships in latent representations with practical impact on robust downstream predictions.

Abstract

Disentangled representation learning aims to learn low-dimensional representations where each dimension corresponds to an underlying generative factor. While the Variational Auto-Encoder (VAE) is widely used for this purpose, most existing methods assume independence among factors, a simplification that does not hold in many real-world scenarios where factors are often interdependent and exhibit causal relationships. To overcome this limitation, we propose the Disentangled Causal Variational Auto-Encoder (DCVAE), a novel supervised VAE framework that integrates causal flows into the representation learning process, enabling the learning of more meaningful and interpretable disentangled representations. We evaluate DCVAE on both synthetic and real-world datasets, demonstrating its superior ability in causal disentanglement and intervention experiments. Furthermore, DCVAE outperforms state-of-the-art methods in various downstream tasks, highlighting its potential for learning true causal structures among factors.

Figures (14)

• Figure 1: Model structure of DCVAE.
• Figure 2: Causal graphs of Pendulum and CelebA. The gray circles represent the causal variables in the graphs. In Figures (a), (b), and (c), we label the underlying factors we are interested in each dataset.
• Figure 3: Results of traverse experiments on Pendulum. Each row corresponds to a variable that we traverse on, specifically, pendulum angle, light angle, shadow length, and shadow position.
• Figure 4: Results of traverse experiments on CelebA(Smile). Each row corresponds to a variable that we traverse on, specifically, smile, gender, cheek bone, mouth open, narrow eye and chubby.
• Figure 5: Results of intervention on only one variable for both Pendulum and CelebA(Smile). The image in the upper left corner of (a) and (b) are the test data we consider respectively.

Theorems & Definitions (1)

• definition 1: Disentangled representation shen2022weakly