Online Time Series Forecasting with Theoretical Guarantees

TL;DR

The paper tackles online time-series forecasting under distribution shifts driven by latent variables. It develops TOT, a theory-backed framework that proves Bayes-risk reductions when forecasters condition on latent states, with guarantees that identifiability of these states improves the risk gap and can be achieved from four consecutive observations. It then proposes a plug-and-play model-agnostic blueprint featuring a temporal decoder, two independent noise estimators for latent and observed transitions, and an ELBO-based training objective with sparsity on the mixing, enabling practical identifiability and forecasting improvements. Empirical results on synthetic and real-world datasets show consistent gains over strong baselines across multiple backbones, supporting both the theoretical guarantees and the practicality of the proposed approach. The work advances causal representation learning in time series and offers a scalable, generalizable path to robust online forecasting in nonstationary environments.

Abstract

This paper is concerned with online time series forecasting, where unknown distribution shifts occur over time, i.e., latent variables influence the mapping from historical to future observations. To develop an automated way of online time series forecasting, we propose a Theoretical framework for Online Time-series forecasting (TOT in short) with theoretical guarantees. Specifically, we prove that supplying a forecaster with latent variables tightens the Bayes risk, the benefit endures under estimation uncertainty of latent variables and grows as the latent variables achieve a more precise identifiability. To better introduce latent variables into online forecasting algorithms, we further propose to identify latent variables with minimal adjacent observations. Based on these results, we devise a model-agnostic blueprint by employing a temporal decoder to match the distribution of observed variables and two independent noise estimators to model the causal inference of latent variables and mixing procedures of observed variables, respectively. Experiment results on synthetic data support our theoretical claims. Moreover, plug-in implementations built on several baselines yield general improvement across multiple benchmarks, highlighting the effectiveness in real-world applications.

Figures (4)

• Figure 1: Illustration of the generation process for time series data, where the mapping of observed variables ${\mathbf{x}}_{t-1}\rightarrow {\mathbf{x}}_t$ is influenced by latent variables ${\mathbf{z}}_t$.
• Figure 2: Illustration of the difference between sparse causal influence and sparse mixing procedure.
• Figure 3: Illustration of the proposed plug-and-play architecture, where the gray blocks (encoder and forecaster) can be replaced with different backbones. The dashed lines denote the backpropagation.
• Figure 4: Ablation study on the Exchange and ETTh2 datasets.

Theorems & Definitions (23)

• Definition 1: Block-wise Identifiability of Latent Variables ${\mathbf{z}}_t$
• Definition 2: Component-wise Identifiability of Latent Variables $z_{t,i}$
• Theorem 1
• Definition 3: Linear Operator hu2008dunford1988linear
• Theorem 2
• Definition 4: Intimate Neighbor Set li2024identificationzhang2024causal
• Theorem 3
• Theorem A1
• proof
• Definition 5