Causal Temporal Representation Learning with Nonstationary Sparse Transition

TL;DR

This work tackles learning causal temporal representations from nonstationary sequences without observed domain indices. It develops identifiability theory showing that, under sparse transition constraints and sufficient variability, domain regimes can be recovered up to label swapping and latent causal processes can be identified up to permutation and component-wise transformations. The CtrlNS framework operationalizes these ideas using a sparse-transition module, a prior network, and a VAE-based encoder-decoder, yielding accurate recovery of distribution shifts and latent factors. Empirically, CtrlNS demonstrates strong identifiability and improved performance on synthetic data and weakly supervised action segmentation benchmarks, highlighting its practical potential for transparent, domain-aware modeling of nonstationary temporal data.

Abstract

Causal Temporal Representation Learning (Ctrl) methods aim to identify the temporal causal dynamics of complex nonstationary temporal sequences. Despite the success of existing Ctrl methods, they require either directly observing the domain variables or assuming a Markov prior on them. Such requirements limit the application of these methods in real-world scenarios when we do not have such prior knowledge of the domain variables. To address this problem, this work adopts a sparse transition assumption, aligned with intuitive human understanding, and presents identifiability results from a theoretical perspective. In particular, we explore under what conditions on the significance of the variability of the transitions we can build a model to identify the distribution shifts. Based on the theoretical result, we introduce a novel framework, Causal Temporal Representation Learning with Nonstationary Sparse Transition (CtrlNS), designed to leverage the constraints on transition sparsity and conditional independence to reliably identify both distribution shifts and latent factors. Our experimental evaluations on synthetic and real-world datasets demonstrate significant improvements over existing baselines, highlighting the effectiveness of our approach.

Figures (8)

• Figure 1: Graphical model for nonstationary causally related time-delayed time-series data with unobserved domain variables $u_t$.
• Figure 2: Illustration of CtrlNS with (1) Sparse Transition, (2) Prior Network, (3) Encoder-Decoder Module.
• Figure 3: Visualization of three phase training process of CtrlNS.
• Figure S1: Illustration of $\hat{\mathcal{C}}$ incorrectly assigning two different domain subsets of inputs $A$ and $B$ into the same $\hat{u}$. The black lines represent the ground truth partition of $\mathcal{C}$ and the orange line represent the incorrect domain partition for set $A$ and $B$.
• Figure S2: Comparison of matrices $\mathcal{M}_i$, $\mathcal{M}_j$, and $\widehat{\mathcal{M}}_k$. The elements in red highlight the differences between them.

Theorems & Definitions (27)

• Definition 1: Observational Equivalence
• Definition 2: Identifiable Domain Variables
• Definition 3: Identifiable Latent Causal Processes
• Definition 4: Matrix Support
• Definition 5: Matrix Function Support
• Definition 6: Weakly Diverse Lossy Transition
• Theorem 1: Identifiability of Domain Variables
• Definition 7: Higher Order Partial Derivative Support Matrix
• Corollary 1: Identifiability under Function Variability
• Lemma 1: Theorem 2 in Yao et al., yao2022temporally