Caformer: Rethinking Time Series Analysis from Causal Perspective

TL;DR

Caformer tackles the challenge of non-stationary time series by explicitly modeling causal relationships between cross-time and cross-dimension dependencies under environmental confounding. It introduces a Structural Causal Model and three dedicated learners—Dependency, Dynamic, and Environment—to perform back-door adjustment and learn robust temporal interactions. The framework demonstrates state-of-the-art performance across forecasting, imputation, classification, and anomaly detection while providing interpretable causal structures. This work offers a principled path to causal time-series analysis with broad applicability and reliable interpretability in real-world environments.

Abstract

Time series analysis is a vital task with broad applications in various domains. However, effectively capturing cross-dimension and cross-time dependencies in non-stationary time series poses significant challenges, particularly in the context of environmental factors. The spurious correlation induced by the environment confounds the causal relationships between cross-dimension and cross-time dependencies. In this paper, we introduce a novel framework called Caformer (\underline{\textbf{Ca}}usal Trans\underline{\textbf{former}}) for time series analysis from a causal perspective. Specifically, our framework comprises three components: Dynamic Learner, Environment Learner, and Dependency Learner. The Dynamic Learner unveils dynamic interactions among dimensions, the Environment Learner mitigates spurious correlations caused by environment with a back-door adjustment, and the Dependency Learner aims to infer robust interactions across both time and dimensions. Our Caformer demonstrates consistent state-of-the-art performance across five mainstream time series analysis tasks, including long- and short-term forecasting, imputation, classification, and anomaly detection, with proper interpretability.

Figures (12)

• Figure 1: Dynamic interactions within time series from a causal perspective. The original time series $\textbf{X}$ comprises four variables, represented by various types of black lines, along with a purple line symbolizing environmental factors. The cross-time dependency is extracted from two time periods with the same patch size $P=5$, while the cross-dimension dependency is inferred from the corresponding cross-time dependency. The structural causal model shows the causality between environmental factors, original time series, cross-dimension and cross-time dependencies.
• Figure 2: The architecture of Caformer (Causal Transformer). It comprises three components: Dependency Learner, Environment Learner, and Dynamic Learner.
• Figure 3: The illustration of patching process and the dependencies among patched time series.
• Figure 4: Comparison between models on five mainstream time series analysis tasks. A larger area indicates a better generalization result of the method across tasks.
• Figure 5: Model comparison in classification. The results are averaged from 10 subsets of UEA.