Causal Representation Learning from Multimodal Biomedical Observations

TL;DR

This work tackles the challenge of learning interpretable, causally structured representations from multimodal biomedical observations under nonparametric latent distributions. It develops a two-stage identifiability framework: first recovering modality-specific latent subspaces, then disentangling them into components by exploiting sparse cross-modal causal connections, all within a flow-based estimation architecture. The primary contributions are formal identifiability theorems for subspaces and components, a practical estimation model enforcing independence and sparsity constraints, and extensive experiments on numerical, synthetic (Variant MNIST), and real-world human phenotype data that align latent factors with biomedical knowledge. The framework advances the reliability and interpretability of multimodal CRL in biomedicine, enabling more trustworthy insights into cross-modal physiological mechanisms and potential clinical decision support.

Abstract

Prevalent in biomedical applications (e.g., human phenotype research), multimodal datasets can provide valuable insights into the underlying physiological mechanisms. However, current machine learning (ML) models designed to analyze these datasets often lack interpretability and identifiability guarantees, which are essential for biomedical research. Recent advances in causal representation learning have shown promise in identifying interpretable latent causal variables with formal theoretical guarantees. Unfortunately, most current work on multimodal distributions either relies on restrictive parametric assumptions or yields only coarse identification results, limiting their applicability to biomedical research that favors a detailed understanding of the mechanisms. In this work, we aim to develop flexible identification conditions for multimodal data and principled methods to facilitate the understanding of biomedical datasets. Theoretically, we consider a nonparametric latent distribution (c.f., parametric assumptions in previous work) that allows for causal relationships across potentially different modalities. We establish identifiability guarantees for each latent component, extending the subspace identification results from previous work. Our key theoretical contribution is the structural sparsity of causal connections between modalities, which, as we will discuss, is natural for a large collection of biomedical systems. Empirically, we present a practical framework to instantiate our theoretical insights. We demonstrate the effectiveness of our approach through extensive experiments on both numerical and synthetic datasets. Results on a real-world human phenotype dataset are consistent with established biomedical research, validating our theoretical and methodological framework.

Figures (8)

• Figure 1: Multimodal data with causal latent variables.
• Figure 2: An illustrative example of the hypothesis space underlying the biomedical system.
• Figure 3: Denser graph $\hat{{\bm{G}}}$.
• Figure 4: Estimation framework. Given multimodal observations ${(\mathbf{x}^{(1)},\hdots,\mathbf{x}^{(M)})}$, the latent variables and domain-specific information in modality $m$ are inferred as $\hat{\mathbf{z}}^{(m)}$ and $\hat{\eta}^{(m)}$ by individual encoders. The observations are then reconstructed with corresponding decoders as $\hat{\mathbf{x}}^{(m)}$. We enforce independence conditions by minimizing the KL divergence term $D_{\text{KL}} \left( [\{\hat{\eta}^{(m)}\}^M_{m=1},\{\hat{\epsilon}_i\}_{i=1}^{d(\mathbf{z})}] || \mathcal{N}(\mathbf{0}, {\bm{I}}) \right)$. We enforce the sparsity constraint by minimizing the $\mathcal{L}_1$ norm in the inferred adjacency matrix $\hat{\mathbf{A}}$.
• Figure 5: Numerical experiment results. (a) Successful recovery of the inter-modal causal graph. (b) Baseline comparisons in different cases. (c) Result of sparsity ablation study.

Theorems & Definitions (22)

• Theorem 4.1: Subspace Identifiability
• Theorem 4.2: Component-wise Identifiability
• Proposition 5.0
• Proposition 5.0
• Proposition B.0
• proof
• Proposition B.1: Independent Noise Condition
• proof
• Theorem C.1: Subspace Identifiability
• proof