Learning to Factorize and Adapt: A Versatile Approach Toward Universal Spatio-Temporal Foundation Models

TL;DR

FactoST-v2 introduces a factorized Spatio-Temporal Foundation Model that decouples universal temporal learning from domain-specific spatial adaptation. It replaces a heavy encoder-decoder with a lightweight encoder-only backbone trained with randomized sequence masking and probabilistic quantile forecasting, then supplements it with a plug-and-play Spatio-Temporal Adapter (STA) comprising STMF, STF, DSPA, and Continual Memory Replay to tailor to new domains with linear complexity. Theoretical analysis argues for tighter generalization bounds and substantial scalability gains, while extensive experiments across eight pretraining domains and multiple ST benchmarks demonstrate state-of-the-art zero-shot, few-shot, and full-shot performance with superior efficiency compared to joint STMs and task-specific baselines. This approach offers a practical, scalable path toward universal STFMs capable of robust cross-domain forecasting with limited data and resources.

Abstract

Spatio-Temporal (ST) Foundation Models (STFMs) promise cross-dataset generalization, yet joint ST pretraining is computationally expensive and grapples with the heterogeneity of domain-specific spatial patterns. Substantially extending our preliminary conference version, we present FactoST-v2, an enhanced factorized framework redesigned for full weight transfer and arbitrary-length generalization. FactoST-v2 decouples universal temporal learning from domain-specific spatial adaptation. The first stage pretrains a minimalist encoder-only backbone using randomized sequence masking to capture invariant temporal dynamics, enabling probabilistic quantile prediction across variable horizons. The second stage employs a streamlined adapter to rapidly inject spatial awareness via meta adaptive learning and prompting. Comprehensive evaluations across diverse domains demonstrate that FactoST-v2 achieves state-of-the-art accuracy with linear efficiency - significantly outperforming existing foundation models in zero-shot and few-shot scenarios while rivaling domain-specific expert baselines. This factorized paradigm offers a practical, scalable path toward truly universal STFMs. Code is available at https://github.com/CityMind-Lab/FactoST.

Figures (13)

• Figure 1: Evolution of Spatio-Temporal Modeling Paradigms. (a) Traditional STGNNs: Coupled modeling tailored for specific graphs. (b) Existing STFMs: Joint pretraining on heterogeneous graphs, limited by topological conflicts. (c) Our Factorized STFM (FactoST): The upper panel highlights the architectural evolution from the rigid, deterministic design of FactoST-v1 (Left) to the flexible, probabilistic backbone of FactoST-v2 (Right). The lower panel illustrates the core Factorized Paradigm—decoupling universal temporal learning (Stage 1) from spatial adaptation (Stage 2).
• Figure 2: Illustration of Spatio-Temporal Modeling. The figure characterizes ST modeling from two perspectives: (Left) Spatio-Temporal Data (Observations): Visualizing the evolution of dynamic signals ($\mathbf{X}$) across spatial nodes and time steps. (Right) Spatio-Temporal Patterns (Dynamics): Depicting the underlying mechanisms, where Temporal Patterns exhibit universal properties over time (e.g., periodicity, trends), while Spatial Patterns manifest in heterogeneous topologies—ranging from Non-Euclidean Graphs (e.g., irregular traffic networks) to Euclidean Grids (e.g., regular raster data).
• Figure 3: Overview of FactoST-v2, consisting of two stages: Universal Temporal Pretraining (UTP) for general temporal feature learning (via instance normalization, semantic-preserving positional encoding, gating attention, and quantile prediction) and Spatio-Temporal Adaptation (STA) for domain-specific adaptation (via metadata fusion, filtering, prompt alignment, and continual memory replay), enabling effective cross-domain transfer for ST sequences.
• Figure 4: Training loss comparison across different model scales. The zoomed-in window details the early training phase. The final converged loss values are annotated on the right, showing a monotonic improvement in performance as model capacity increases from Minuscule to Base.
• Figure 5: Ablation of STA's components on PEMS-03 short-term forecasting, showing MAE/RMSE degradation upon removing key modules.