Emerging Synergies in Causality and Deep Generative Models: A Survey

TL;DR

The paper tackles the challenge of building more interpretable and generalizable generative systems by tying together causality and deep generative models (DGMs). It presents a two-pronged agenda: (i) integrating causal principles into DGMs (via GANs, VAEs, and diffusion) to create causal DGMs (CGMs) and to enable interventions and counterfactuals, and (ii) using DGMs to identify and reason about causal structures, including causal discovery, counterfactual inference, and representation learning. A substantial portion is devoted to how large-scale DGMs, especially generative LLMs, can participate in causal tasks, including evaluating memorization versus reasoning, prompting strategies, and data-driven pipelines that combine LLMs with traditional causal methods. The survey covers trustworthy properties (generalization, fairness, interpretability) and applications in healthcare, social science, and interdisciplinary domains, while outlining open challenges (scalability to large causal graphs, identifiability, evaluation of counterfactuals) and future directions. Overall, the work provides a comprehensive guide to a rapidly evolving area at the intersection of causality and generative modeling, with practical implications for robust and interpretable AI systems.

Abstract

In the field of artificial intelligence (AI), the quest to understand and model data-generating processes (DGPs) is of paramount importance. Deep generative models (DGMs) have proven adept in capturing complex data distributions but often fall short in generalization and interpretability. On the other hand, causality offers a structured lens to comprehend the mechanisms driving data generation and highlights the causal-effect dynamics inherent in these processes. While causality excels in interpretability and the ability to extrapolate, it grapples with intricacies of high-dimensional spaces. Recognizing the synergistic potential, we delve into the confluence of causality and DGMs. We elucidate the integration of causal principles within DGMs, investigate causal identification using DGMs, and navigate an emerging research frontier of causality in large-scale generative models, particularly generative large language models (LLMs). We offer insights into methodologies, highlight open challenges, and suggest future directions, positioning our comprehensive review as an essential guide in this swiftly emerging and evolving area.

Figures (10)

• Figure 1: Comparison of data-generating processes within deep generative models (DGMs) and causality. DGMs primarily draw latent variables from a simple distribution such as Gaussian Goodfellow2014GenerativeANKingma2014AutoEncodingVB, while causality is rooted in variables defined by causal relationships von2021self. This divergence in foundation grants causality superior extrapolation and interpretability, and DGMs an edge in managing high-dimensional spaces, indicating synergistic opportunities.
• Figure 2: The illustration of the intersections of causality and deep generative models (DGMs). § \ref{['971699495725']} examines causal principles in DGMs, § \ref{['776815666820']} explores DGMs in causal tasks, § \ref{['605156580006']} discusses causality in large-scale DGMs, § \ref{['471209836872']} addresses enhancing DGM trustworthiness through causality, § \ref{['591853874791']} covers varied applications of causality with DGMs, and § \ref{['996252365804']} provides potential directions.
• Figure 3: In the CausalGAN framework Kocaoglu2018CausalGANLC, the depicted causal graph is based on a subset of the CelebFaces Attributes Dataset (CelebA) liu2015faceattributes. This graph demonstrates how DGMs can isolate the causal influences of $Age$ and $Gender$ on the $Mustache$ attribute when performing interventions. Notably, both male and female faces appear when sampling from the interventional distribution with $Mustache=1$, whereas only male faces are observed when sampled from the conditional distribution of $Mustache=1$ since $P(Male=1\vert Mustache=1)=1$.
• Figure 4: CausalGAN Kocaoglu2018CausalGANLC illustrates the process of translating an SCM into the structure of a generator architecture using a simplified causal graph $\{A\rightarrow X \leftarrow B\}$weng2021diffusion.
• Figure 5: The illustration of encoding an SCM as the prior for latent variables in bidirectional generative models shen2022weaklyweng2021diffusion.

Theorems & Definitions (1)

• Definition 1