Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Causally Disentangled Representations via the Principle of Independent Causal Mechanisms

Aneesh Komanduri, Yongkai Wu, Feng Chen, Xintao Wu

TL;DR

This work defines a new notion of causal disentanglement from the perspective of independent causal mechanisms, and proposes ICM-VAE, a framework for learning causally disentangled representations supervised by causally related observed labels.

Abstract

Learning disentangled causal representations is a challenging problem that has gained significant attention recently due to its implications for extracting meaningful information for downstream tasks. In this work, we define a new notion of causal disentanglement from the perspective of independent causal mechanisms. We propose ICM-VAE, a framework for learning causally disentangled representations supervised by causally related observed labels. We model causal mechanisms using nonlinear learnable flow-based diffeomorphic functions to map noise variables to latent causal variables. Further, to promote the disentanglement of causal factors, we propose a causal disentanglement prior learned from auxiliary labels and the latent causal structure. We theoretically show the identifiability of causal factors and mechanisms up to permutation and elementwise reparameterization. We empirically demonstrate that our framework induces highly disentangled causal factors, improves interventional robustness, and is compatible with counterfactual generation.

Learning Causally Disentangled Representations via the Principle of Independent Causal Mechanisms

TL;DR

This work defines a new notion of causal disentanglement from the perspective of independent causal mechanisms, and proposes ICM-VAE, a framework for learning causally disentangled representations supervised by causally related observed labels.

Abstract

Learning disentangled causal representations is a challenging problem that has gained significant attention recently due to its implications for extracting meaningful information for downstream tasks. In this work, we define a new notion of causal disentanglement from the perspective of independent causal mechanisms. We propose ICM-VAE, a framework for learning causally disentangled representations supervised by causally related observed labels. We model causal mechanisms using nonlinear learnable flow-based diffeomorphic functions to map noise variables to latent causal variables. Further, to promote the disentanglement of causal factors, we propose a causal disentanglement prior learned from auxiliary labels and the latent causal structure. We theoretically show the identifiability of causal factors and mechanisms up to permutation and elementwise reparameterization. We empirically demonstrate that our framework induces highly disentangled causal factors, improves interventional robustness, and is compatible with counterfactual generation.
Paper Structure (33 sections, 3 theorems, 64 equations, 9 figures, 2 tables)

This paper contains 33 sections, 3 theorems, 64 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work.
  3. Contributions.
  4. Preliminaries
  5. Identifiability
  6. Structural Causal Model
  7. Causal Mechanism Equivalence
  8. Proposed Framework
  9. Structural Causal Flow
  10. Generative Model
  11. Causal Disentanglement Prior
  12. Learning Objective
  13. Identifiability Analysis
  14. Experimental Evaluation
  15. Datasets.
  16. ...and 18 more sections

Key Result

Theorem 1

Suppose that we observe data sampled from a generative model defined according to (generative_model)-(eq:disentanglement_prior_unit) with two sets of model parameters $\theta = (g, \mathbf{T}, \bm{\lambda}, G^z)$ and $\hat{\theta} = (\hat{g}, \hat{\mathbf{T}}, \hat{\bm{\lambda}}, \hat{G}^z)$. Suppos Then $\theta$ and $\hat{\theta}$ are causal mechanism permutation-equivalent, and the model $\hat{\

Figures (9)

  • Figure 1: We learn causal models representing images as latent causal variables $z$. The bottom shows the effect of intervening on the latent code corresponding to the pendulum's angle, propagating effects, and generating a counterfactual image.
  • Figure 2: Architecture of ICM-VAE Framework, which contains two main components: (i) Structural Causal Flow (SCF), and (ii) Causal Disentanglement Prior. The blue color represents prior components and the orange represents the learning process.
  • Figure 3: Pendulum (left) and Flow (right) counterfactual images after intervening on causal factors, individually, and propagating effects.
  • Figure 4: CausalCircuit counterfactual images by intervening on robot arm to turn on a colored light, which can causally affect other lights.
  • Figure 5: Counterexample to traditional disentanglement
  • ...and 4 more figures

Theorems & Definitions (13)

  • Definition 1: $\sim$-identifiability
  • Definition 2: pmlr-v108-khemakhem20a
  • Definition 3: pmlr-v108-khemakhem20a
  • Definition 4: Permutation Disentanglement
  • Definition 5: Causal Mechanism Permutation Equivalence
  • Definition 6: Causal Disentanglement
  • Theorem 1: Identifiability of ICM-VAE
  • Definition 7: Minimal Sufficient Statistic lachapelle2022disentanglement
  • Definition 8: Permutation-Scaling Matrix lachapelle2022disentanglement
  • Lemma 1: lachapelle2022disentanglement
  • ...and 3 more