Time Series Domain Adaptation via Latent Invariant Causal Mechanism

TL;DR

The paper tackles time series domain adaptation under distribution shift by shifting focus from observed variables to latent causal mechanisms. It introduces Latent Causality Alignment (LCA), a variational framework with a flow-based prior and sparsity constraints to uncover latent Granger causal structures and align them across domains, with identifiability guarantees. The method yields significant improvements on forecasting and classification tasks across eight datasets, including high-dimensional video data, and ablation studies confirm the necessity of each component. This latent-representation approach advances causal representation learning in time series and offers practical gains for real-world domain adaptation challenges.

Abstract

Time series domain adaptation aims to transfer the complex temporal dependence from the labeled source domain to the unlabeled target domain. Recent advances leverage the stable causal mechanism over observed variables to model the domain-invariant temporal dependence. However, modeling precise causal structures in high-dimensional data, such as videos, remains challenging. Additionally, direct causal edges may not exist among observed variables (e.g., pixels). These limitations hinder the applicability of existing approaches to real-world scenarios. To address these challenges, we find that the high-dimension time series data are generated from the low-dimension latent variables, which motivates us to model the causal mechanisms of the temporal latent process. Based on this intuition, we propose a latent causal mechanism identification framework that guarantees the uniqueness of the reconstructed latent causal structures. Specifically, we first identify latent variables by utilizing sufficient changes in historical information. Moreover, by enforcing the sparsity of the relationships of latent variables, we can achieve identifiable latent causal structures. Built on the theoretical results, we develop the Latent Causality Alignment (LCA) model that leverages variational inference, which incorporates an intra-domain latent sparsity constraint for latent structure reconstruction and an inter-domain latent sparsity constraint for domain-invariant structure reconstruction. Experiment results on eight benchmarks show a general improvement in the domain-adaptive time series classification and forecasting tasks, highlighting the effectiveness of our method in real-world scenarios. Codes are available at https://github.com/DMIRLAB-Group/LCA.

Figures (8)

• Figure 1: A toy domain adaptation example for video data, where the blue arrows denote the estimated causal relationships. (a) Two three-frame videos, one featuring a walking human and the other a monster, represent the source and target domains, respectively. (b) Since the skeleton variables are latent confounders and there are no causal relationships among pixels, the existing methods may learn dense and fault causal relationships among adjacent two frames, failing to extract the domain-invariant causal mechanism. (c) By identifying the latent variables like the joints in the skeleton, it is convenient to model the latent invariant causal mechanisms behind observed variables. Hence, we can address the time series domain adaptation problem in complex real-world scenarios.
• Figure 2: Model architecture: Blue arrows indicate the flow of source data, while purple arrows represent the flow of target data. The loss function is highlighted in bold as $L$. Subfigures (a) and (b) depict the architectures for prediction and classification tasks, respectively.
• Figure 3: To describe the relationship between partial derivatives and the existence of edges clearly, we assume that the data follow a simple linear generation process.
• Figure 4: The MSE and MAE values after predicting different lengths in the transition from domain 1 to domain 2 in the ETT dataset. Subfigures (a), (b), (c), and (d) show the forecasting results on lengths of 10, 20, 30, and 40 time steps, respectively.
• Figure 5: Classification Visualization: (a) and (b) show scatter plots of the latent variables and probability vectors, respectively, after dimensionality reduction using t-SNE for the H$\rightarrow$U scenario. (c) and (d) present the corresponding scatter plots for the U$\rightarrow$H scenario. These visualizations illustrate how the model’s latent representations and probability vectors effectively capture and distinguish between categories.

Theorems & Definitions (8)

• Definition 1: Identifiable Latent Causal Process
• Lemma 1
• Proposition 2
• Lemma 3
• proof
• Definition 2
• Proposition 4
• proof