Disentangling Long-Short Term State Under Unknown Interventions for Online Time Series Forecasting

TL;DR

This work tackles online time series forecasting under nonstationarity caused by unknown interventions on short-term latent factors. It develops a data-generation model and an identifiability framework that proves long-term and short-term latent states can be disentangled up to invertible mappings. The Long Short-Term Disentanglement (LSTD) model implements separate encoders and priors for long/short-term states and imposes a smooth constraint for long-term retention and an interrupted dependency constraint for short-term forgetting, enabling robust online adaptation. Empirical results across diverse benchmarks show substantial improvements over state-of-the-art online methods, underscoring the practical value of causal representation learning for nonstationary environments.

Abstract

Current methods for time series forecasting struggle in the online scenario, since it is difficult to preserve long-term dependency while adapting short-term changes when data are arriving sequentially. Although some recent methods solve this problem by controlling the updates of latent states, they cannot disentangle the long/short-term states, leading to the inability to effectively adapt to nonstationary. To tackle this challenge, we propose a general framework to disentangle long/short-term states for online time series forecasting. Our idea is inspired by the observations where short-term changes can be led by unknown interventions like abrupt policies in the stock market. Based on this insight, we formalize a data generation process with unknown interventions on short-term states. Under mild assumptions, we further leverage the independence of short-term states led by unknown interventions to establish the identification theory to achieve the disentanglement of long/short-term states. Built on this theory, we develop a long short-term disentanglement model (LSTD) to extract the long/short-term states with long/short-term encoders, respectively. Furthermore, the LSTD model incorporates a smooth constraint to preserve the long-term dependencies and an interrupted dependency constraint to enforce the forgetting of short-term dependencies, together boosting the disentanglement of long/short-term states. Experimental results on several benchmark datasets show that our \textbf{LSTD} model outperforms existing methods for online time series forecasting, validating its efficacy in real-world applications.

Figures (7)

• Figure 1: Illustration of sequentially arriving exchange rate data, which is influenced by short-term customs duties and long-term financial revenue. Moreover, the short-term customs duties are intervened by sudden customs tariff policies. (a) If the estimated short-term customs duties and long-term financial revenue are entangled, short-term influence from Environments (e.g., $e_1, e_2, e_3$) may affect the effectiveness of the models to adapt to the changing environments, leading to suboptimal forecasting performance. (b) When the long/short-term states are disentangled, the model can quickly adapt to environmental changes and hence achieve correct forecasting results. (c) Data generation process for time-series data. $\bm{z}_{t}^s$ and $\bm{z}_{t}^d$ denotes the long/short-term states. Note that the short-term states $\bm{z}_{t}^d$ are intervened randomly.
• Figure 2: The framework of the proposed LSTD model. The long/short-term latent variables ${\mathbf{z}}_{1:L}^d$ and ${\mathbf{z}}_{1:L}^s$ are extracted from the encoder. And the latent transition module is used to estimated the ${\mathbf{z}}_{L+1:H}^d$ and the ${\mathbf{z}}_{L+1:H}^s$ from ${\mathbf{z}}_{1:L}^d$ and ${\mathbf{z}}_{1:L}^s$, respectively. The long-term and short-term prior networks are used to estimate the prior distributions.
• Figure 3: Ablation study on the Exchange datasets. We explore the impact of different loss terms
• Figure 4: The figure (a) represents the visualization of the proposed LSTD and other baselines. The blue lines denote the ground-truth time series data and the lines with other colors denote the predicted results of different methods. The figure (b) shows the visualization of the LSTD method for detecting interventions. The yellow lines represent the real-time series data, and the red lines represent the gradient. Black dotted lines denote intervention occurs. (Best view in color)
• Figure 5: All ablation experiments

Theorems & Definitions (3)

• Theorem 1
• Theorem 2
• proof