Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Unsupervised Causal Representation Learning via Latent Additive Noise Model Causal Autoencoders

Hans Jarett J. Ong, Brian Godwin S. Lim, Dominic Dayta, Renzo Roel P. Tan, Kazushi Ikeda

TL;DR

LANCA introduces a latent Additive Noise Model (ANM) causal autoencoder that replaces stochastic VAE encoding with a deterministic Wasserstein Auto-Encoder, enabling explicit optimization of latent residual independence. By learning a differentiable ANM layer and a DAG-structured graph, LANCA identifies latent causal factors under ANM constraints, achieving component-wise identifiability up to affine transformations and robust performance on synthetic physics tasks and the complex CANDLE dataset. The method demonstrates superior unsupervised causal structure learning, interventional robustness to confounding, and favorable unconfoundedness and counterfactual generativity, while highlighting theoretical limits that pure observational ANMs cannot guarantee full identifiability under general nonlinear mixing. LANCA thus offers a practical, scalable inductive bias for unsupervised causal discovery in rich visual domains, with strong implications for robust representation learning and counterfactual reasoning in real-world settings.

Abstract

Unsupervised representation learning seeks to recover latent generative factors, yet standard methods relying on statistical independence often fail to capture causal dependencies. A central challenge is identifiability: as established in disentangled representation learning and nonlinear ICA literature, disentangling causal variables from observational data is impossible without supervision, auxiliary signals, or strong inductive biases. In this work, we propose the Latent Additive Noise Model Causal Autoencoder (LANCA) to operationalize the Additive Noise Model (ANM) as a strong inductive bias for unsupervised discovery. Theoretically, we prove that while the ANM constraint does not guarantee unique identifiability in the general mixing case, it resolves component-wise indeterminacy by restricting the admissible transformations from arbitrary diffeomorphisms to the affine class. Methodologically, arguing that the stochastic encoding inherent to VAEs obscures the structural residuals required for latent causal discovery, LANCA employs a deterministic Wasserstein Auto-Encoder (WAE) coupled with a differentiable ANM Layer. This architecture transforms residual independence from a passive assumption into an explicit optimization objective. Empirically, LANCA outperforms state-of-the-art baselines on synthetic physics benchmarks (Pendulum, Flow), and on photorealistic environments (CANDLE), where it demonstrates superior robustness to spurious correlations arising from complex background scenes.

Towards Unsupervised Causal Representation Learning via Latent Additive Noise Model Causal Autoencoders

TL;DR

LANCA introduces a latent Additive Noise Model (ANM) causal autoencoder that replaces stochastic VAE encoding with a deterministic Wasserstein Auto-Encoder, enabling explicit optimization of latent residual independence. By learning a differentiable ANM layer and a DAG-structured graph, LANCA identifies latent causal factors under ANM constraints, achieving component-wise identifiability up to affine transformations and robust performance on synthetic physics tasks and the complex CANDLE dataset. The method demonstrates superior unsupervised causal structure learning, interventional robustness to confounding, and favorable unconfoundedness and counterfactual generativity, while highlighting theoretical limits that pure observational ANMs cannot guarantee full identifiability under general nonlinear mixing. LANCA thus offers a practical, scalable inductive bias for unsupervised causal discovery in rich visual domains, with strong implications for robust representation learning and counterfactual reasoning in real-world settings.

Abstract

Unsupervised representation learning seeks to recover latent generative factors, yet standard methods relying on statistical independence often fail to capture causal dependencies. A central challenge is identifiability: as established in disentangled representation learning and nonlinear ICA literature, disentangling causal variables from observational data is impossible without supervision, auxiliary signals, or strong inductive biases. In this work, we propose the Latent Additive Noise Model Causal Autoencoder (LANCA) to operationalize the Additive Noise Model (ANM) as a strong inductive bias for unsupervised discovery. Theoretically, we prove that while the ANM constraint does not guarantee unique identifiability in the general mixing case, it resolves component-wise indeterminacy by restricting the admissible transformations from arbitrary diffeomorphisms to the affine class. Methodologically, arguing that the stochastic encoding inherent to VAEs obscures the structural residuals required for latent causal discovery, LANCA employs a deterministic Wasserstein Auto-Encoder (WAE) coupled with a differentiable ANM Layer. This architecture transforms residual independence from a passive assumption into an explicit optimization objective. Empirically, LANCA outperforms state-of-the-art baselines on synthetic physics benchmarks (Pendulum, Flow), and on photorealistic environments (CANDLE), where it demonstrates superior robustness to spurious correlations arising from complex background scenes.
Paper Structure (60 sections, 2 theorems, 11 equations, 2 figures, 6 tables)

This paper contains 60 sections, 2 theorems, 11 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Disentangled Representation Learning and the Limits of Independence
  4. Causal Representation Learning
  5. Identifiability via Additive Noise Models
  6. Identifiability Analysis
  7. Identifiability under Component-Wise Distortion
  8. Impossibility Result for General Nonlinear Mixing
  9. Methodology
  10. Deterministic Encoding via Wasserstein Auto-Encoders
  11. The Differentiable ANM Layer
  12. Differentiable DAG via Matrix Factorization.
  13. Mechanism Learning and Structural Abduction.
  14. Optimization Objective
  15. Structural Consistency Reconstruction ($\mathcal{L}_{\text{recon}}$).
  16. ...and 45 more sections

Key Result

Theorem 1

Let $s_i$ be an endogenous ground-truth variable (i.e., $\mathbf{pa}_i \neq \emptyset$) generated by a valid ANM $s_i = h_i(\mathbf{pa}_i) + n_i$, where $h_i$ is a non-constant, differentiable function of the parents $\mathbf{pa}_i$. Let $z_i = \psi_i(s_i)$ be a smooth, invertible transformation of

Figures (2)

  • Figure 1: The Latent ANM Causal Autoencoder (LANCA). (A) The encoder $\mathbf{E}_\phi$ maps $\mathbf{x}$ to a deterministic latent code $\mathbf{z}$ via the WAE framework. (B) The ANM Layer learns the adjacency matrix $\mathbf{A} = \mathbf{\Pi}^\top \mathbf{U} \mathbf{\Pi}$, the set of structural equations $\{f_i\}_{i=1}^n$, and performs structural abduction to recover the residual vector $\boldsymbol{\hat{\epsilon}}$. (C) The objective enforces reconstruction accuracy on both the direct encoding path and the ANM-regenerated path. (D) The model matches the joint distribution of residuals $\mathrm{Q}(\boldsymbol{\hat{\epsilon}})$ to a factorized prior $\mathrm{P}(\boldsymbol{\epsilon})$ via Maximum Mean Discrepancy (MMD). (E) Counterfactual generation is achieved via the three-step procedure: abduction of $\boldsymbol{\hat{\epsilon}}$, intervention on specific factors $do(z_i=c)$ for some constant $c$, and prediction of the new observation $\mathbf{x}_\text{cf}=\mathbf{D}_\theta(\mathbf{z}_\text{cf})$.
  • Figure 2: Qualitative Counterfactual Generation. Comparison of counterfactual images generated by LANCA (Ours), CausalVAE (Supervised), and $\beta$-VAE.

Theorems & Definitions (2)

  • Theorem 1: Linearization of Endogenous Variables
  • Proposition 1: Existence of Spurious Independent Solutions