Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Expand and Compress: Exploring Tuning Principles for Continual Spatio-Temporal Graph Forecasting

Wei Chen, Yuxuan Liang

TL;DR

This paper tackles continual spatio-temporal forecasting on dynamically expanding graphs by freezing the backbone STGNN and learning a memory-based continuous prompt pool. Guided by two principles—expand to accommodate heterogeneity and compress to control parameter growth—the method EAC uses node-level prompts that adapt to streaming data without full-backbone retraining. The approach achieves superior accuracy, universality across diverse STGNN backbones, and improved efficiency, while keeping the tuning lightweight through low-rank compression. Empirical results on Air-Stream, PEMS-Stream, and Energy-Stream demonstrate practical impact for real-world streaming forecasting tasks with growing sensor networks. The work suggests promising directions toward foundation-level spatio-temporal models and scalable pre-training for continual forecasting.

Abstract

The widespread deployment of sensing devices leads to a surge in data for spatio-temporal forecasting applications such as traffic flow, air quality, and wind energy. Although spatio-temporal graph neural networks have achieved success in modeling various static spatio-temporal forecasting scenarios, real-world spatio-temporal data are typically received in a streaming manner, and the network continuously expands with the installation of new sensors. Thus, spatio-temporal forecasting in streaming scenarios faces dual challenges: the inefficiency of retraining models over newly arrived data and the detrimental effects of catastrophic forgetting over long-term history. To address these challenges, we propose a novel prompt tuning-based continuous forecasting method, following two fundamental tuning principles guided by empirical and theoretical analysis: expand and compress, which effectively resolve the aforementioned problems with lightweight tuning parameters. Specifically, we integrate the base spatio-temporal graph neural network with a continuous prompt pool, utilizing stored prompts (i.e., few learnable parameters) in memory, and jointly optimize them with the base spatio-temporal graph neural network. This method ensures that the model sequentially learns from the spatio-temporal data stream to accomplish tasks for corresponding periods. Extensive experimental results on multiple real-world datasets demonstrate the multi-faceted superiority of our method over the state-of-the-art baselines, including effectiveness, efficiency, universality, etc.

Expand and Compress: Exploring Tuning Principles for Continual Spatio-Temporal Graph Forecasting

TL;DR

This paper tackles continual spatio-temporal forecasting on dynamically expanding graphs by freezing the backbone STGNN and learning a memory-based continuous prompt pool. Guided by two principles—expand to accommodate heterogeneity and compress to control parameter growth—the method EAC uses node-level prompts that adapt to streaming data without full-backbone retraining. The approach achieves superior accuracy, universality across diverse STGNN backbones, and improved efficiency, while keeping the tuning lightweight through low-rank compression. Empirical results on Air-Stream, PEMS-Stream, and Energy-Stream demonstrate practical impact for real-world streaming forecasting tasks with growing sensor networks. The work suggests promising directions toward foundation-level spatio-temporal models and scalable pre-training for continual forecasting.

Abstract

The widespread deployment of sensing devices leads to a surge in data for spatio-temporal forecasting applications such as traffic flow, air quality, and wind energy. Although spatio-temporal graph neural networks have achieved success in modeling various static spatio-temporal forecasting scenarios, real-world spatio-temporal data are typically received in a streaming manner, and the network continuously expands with the installation of new sensors. Thus, spatio-temporal forecasting in streaming scenarios faces dual challenges: the inefficiency of retraining models over newly arrived data and the detrimental effects of catastrophic forgetting over long-term history. To address these challenges, we propose a novel prompt tuning-based continuous forecasting method, following two fundamental tuning principles guided by empirical and theoretical analysis: expand and compress, which effectively resolve the aforementioned problems with lightweight tuning parameters. Specifically, we integrate the base spatio-temporal graph neural network with a continuous prompt pool, utilizing stored prompts (i.e., few learnable parameters) in memory, and jointly optimize them with the base spatio-temporal graph neural network. This method ensures that the model sequentially learns from the spatio-temporal data stream to accomplish tasks for corresponding periods. Extensive experimental results on multiple real-world datasets demonstrate the multi-faceted superiority of our method over the state-of-the-art baselines, including effectiveness, efficiency, universality, etc.

Paper Structure

This paper contains 34 sections, 2 theorems, 32 equations, 8 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Methodology
  5. Expand: Heterogeneity-guided Continuous Prompt Pool Growth
  6. Compress: Low-rank-guided Continuous Prompt Pool Reduction
  7. Experiments
  8. Experimental setup
  9. Effectiveness Study (RQ1)
  10. Universality Study (RQ2)
  11. Efficiency & Lightweight Study (RQ3 & RQ4)
  12. Simplicity Study (RQ5)
  13. Conclusion
  14. Theoretical Proof
  15. Prompt Parameter Pool Can Continuously Adapt to Heterogeneity Property
  16. ...and 19 more sections

Key Result

Proposition 1

For an original node input feature matrix $X = [x_1, \cdots, x_n] \in \mathbb{R}^{n \times d}$, we introduce a node prompt parameter matrix $P = [p_1, \cdots, p_n] \in \mathbb{R}^{n \times d}$. Through a spatio-temporal learning function $f_\theta$ with invariance, a new feature matrix $X^{\theta} = where $P^{\theta} = [p^{\theta}_1, \cdots, p^{\theta}_n] \in \mathbb{R}^{n \times d}$ represents th

Figures (8)

  • Figure 1: Comparison of classic schemes and EAC for continual spatio-temporal forecasting.
  • Figure 2: The overall architecture of our proposed EAC .
  • Figure 3: Heterogeneity measurement.
  • Figure 4: Low-rank measurement.
  • Figure 5: Few-Shot Scenario Forecasting in PEMS-Stream benchmark.
  • ...and 3 more figures

Theorems & Definitions (6)

  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • proof
  • proof