Structural Knowledge Informed Continual Multivariate Time Series Forecasting

TL;DR

The paper tackles continual multivariate time series forecasting under regime shifts, where maintaining accurate variable dependencies over time is challenging due to catastrophic forgetting. It introduces SKI-CL, a framework that learns dynamic graphs per regime while enforcing consistency with external structural knowledge through a structure-aware regularizer, and preserves temporal and dependency information via a representation-matching memory replay scheme. Key innovations include a dynamic graph inference module, adaptive handling of binary/continuous structural priors, a TGConv-based forecasting backbone, and CORAL-driven mode-based sample selection to maximize regime coverage. Empirical results on synthetic and real datasets show that SKI-CL yields superior average performance and better preservation of learned structures compared to state-of-the-art baselines, demonstrating practical impact for reliable forecasting and interpretable dependency inference across regimes.

Abstract

Recent studies in multivariate time series (MTS) forecasting reveal that explicitly modeling the hidden dependencies among different time series can yield promising forecasting performance and reliable explanations. However, modeling variable dependencies remains underexplored when MTS is continuously accumulated under different regimes (stages). Due to the potential distribution and dependency disparities, the underlying model may encounter the catastrophic forgetting problem, i.e., it is challenging to memorize and infer different types of variable dependencies across different regimes while maintaining forecasting performance. To address this issue, we propose a novel Structural Knowledge Informed Continual Learning (SKI-CL) framework to perform MTS forecasting within a continual learning paradigm, which leverages structural knowledge to steer the forecasting model toward identifying and adapting to different regimes, and selects representative MTS samples from each regime for memory replay. Specifically, we develop a forecasting model based on graph structure learning, where a consistency regularization scheme is imposed between the learned variable dependencies and the structural knowledge while optimizing the forecasting objective over the MTS data. As such, MTS representations learned in each regime are associated with distinct structural knowledge, which helps the model memorize a variety of conceivable scenarios and results in accurate forecasts in the continual learning context. Meanwhile, we develop a representation-matching memory replay scheme that maximizes the temporal coverage of MTS data to efficiently preserve the underlying temporal dynamics and dependency structures of each regime. Thorough empirical studies on synthetic and real-world benchmarks validate SKI-CL's efficacy and advantages over the state-of-the-art for continual MTS forecasting tasks.

Figures (11)

• Figure 1: An illustration depicting the catastrophic forgetting of learned dependency structures (i.e., the interactions of variables) in multivariate time series forecasting across regimes. Each regime is characterized by a distinct operational logic of the system.
• Figure 2: The proposed SKI-CL framework for continual MTS forecasting. The training objectives for each regime contains the current training data and the memory buffer. After training at each regime, the structural knowledge and samples selected by our representation-matching scheme are added to the current memory. At testing phase, SKI-CL is able to dynamically infer faithful dependency structures for different regimes without accessing the memory buffer.
• Figure 3: The proposed SKI-CL Model for dependencies modeling and MTS forecasting.
• Figure 4: The Structure Visualizations on Traffic-CL and Solar-CL datasets.
• Figure 5: Model Performance with and without Memory Replay (Lower MAE and RMSE indicate better forecasting performance; higher Precision and Recall indicate higher structure similarity.