Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decoupling Spatio-Temporal Prediction: When Lightweight Large Models Meet Adaptive Hypergraphs

Jiawen Chen, Qi Shao, Duxin Chen, Wenwu Yu

TL;DR

This work tackles the challenge of achieving accurate spatio-temporal predictions at scale by decoupling temporal and spatial modeling. It introduces STH-SepNet, which uses lightweight large language models to capture low-rank temporal dynamics and adaptive hypergraphs to model evolving, higher-order spatial interactions, fused through a gating mechanism. The approach demonstrates state-of-the-art predictive performance on large real-world datasets and delivers notable computational efficiency, outperforming many large-model baselines. The results underscore the practical value of separating temporal and spatial processing for scalable, robust spatio-temporal forecasting in dynamic networks.

Abstract

Spatio-temporal prediction is a pivotal task with broad applications in traffic management, climate monitoring, energy scheduling, etc. However, existing methodologies often struggle to balance model expressiveness and computational efficiency, especially when scaling to large real-world datasets. To tackle these challenges, we propose STH-SepNet (Spatio-Temporal Hypergraph Separation Networks), a novel framework that decouples temporal and spatial modeling to enhance both efficiency and precision. Therein, the temporal dimension is modeled using lightweight large language models, which effectively capture low-rank temporal dynamics. Concurrently, the spatial dimension is addressed through an adaptive hypergraph neural network, which dynamically constructs hyperedges to model intricate, higher-order interactions. A carefully designed gating mechanism is integrated to seamlessly fuse temporal and spatial representations. By leveraging the fundamental principles of low-rank temporal dynamics and spatial interactions, STH-SepNet offers a pragmatic and scalable solution for spatio-temporal prediction in real-world applications. Extensive experiments on large-scale real-world datasets across multiple benchmarks demonstrate the effectiveness of STH-SepNet in boosting predictive performance while maintaining computational efficiency. This work may provide a promising lightweight framework for spatio-temporal prediction, aiming to reduce computational demands and while enhancing predictive performance. Our code is avaliable at https://github.com/SEU-WENJIA/ST-SepNet-Lightweight-LLMs-Meet-Adaptive-Hypergraphs.

Decoupling Spatio-Temporal Prediction: When Lightweight Large Models Meet Adaptive Hypergraphs

TL;DR

This work tackles the challenge of achieving accurate spatio-temporal predictions at scale by decoupling temporal and spatial modeling. It introduces STH-SepNet, which uses lightweight large language models to capture low-rank temporal dynamics and adaptive hypergraphs to model evolving, higher-order spatial interactions, fused through a gating mechanism. The approach demonstrates state-of-the-art predictive performance on large real-world datasets and delivers notable computational efficiency, outperforming many large-model baselines. The results underscore the practical value of separating temporal and spatial processing for scalable, robust spatio-temporal forecasting in dynamic networks.

Abstract

Spatio-temporal prediction is a pivotal task with broad applications in traffic management, climate monitoring, energy scheduling, etc. However, existing methodologies often struggle to balance model expressiveness and computational efficiency, especially when scaling to large real-world datasets. To tackle these challenges, we propose STH-SepNet (Spatio-Temporal Hypergraph Separation Networks), a novel framework that decouples temporal and spatial modeling to enhance both efficiency and precision. Therein, the temporal dimension is modeled using lightweight large language models, which effectively capture low-rank temporal dynamics. Concurrently, the spatial dimension is addressed through an adaptive hypergraph neural network, which dynamically constructs hyperedges to model intricate, higher-order interactions. A carefully designed gating mechanism is integrated to seamlessly fuse temporal and spatial representations. By leveraging the fundamental principles of low-rank temporal dynamics and spatial interactions, STH-SepNet offers a pragmatic and scalable solution for spatio-temporal prediction in real-world applications. Extensive experiments on large-scale real-world datasets across multiple benchmarks demonstrate the effectiveness of STH-SepNet in boosting predictive performance while maintaining computational efficiency. This work may provide a promising lightweight framework for spatio-temporal prediction, aiming to reduce computational demands and while enhancing predictive performance. Our code is avaliable at https://github.com/SEU-WENJIA/ST-SepNet-Lightweight-LLMs-Meet-Adaptive-Hypergraphs.

Paper Structure

This paper contains 38 sections, 1 theorem, 18 equations, 7 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Large Models for Prediction
  4. Spatio-Temporal Prediction
  5. Preliminaries
  6. Problem Formulation
  7. Adaptive Network Construction
  8. Incident Matrix
  9. Adaptive HyperGraph Construction
  10. Methodology
  11. Global Trend Module
  12. Prompt Adaptation Module
  13. Hypergraph Spatio-temporal Module
  14. Gated Fusion Module
  15. Experiments
  16. ...and 23 more sections

Key Result

Theorem 1

For any $k \geq 2$, the $(k-1)$-hops neighborhood of a node $v$, denoted as $N_{k-1} (v)$, corresponds to all nodes involved in the $k$-order hyperedges in $H_v^k$, if and only if the following conditions are satisfied: For each $w\in N_{k-1} (v)$, (1) Local Connectivity Condition: there exists at l where $F_1, F_2, \ldots, u_k$ are intermediary nodes.

Figures (7)

  • Figure 1: (a) Spatio-temporal data exhibit spatial distribution shifts across different nodes. (b) Dynamic adaptive hypergraph captures evolving spatial distribution patterns.
  • Figure 2: The framework of STH-SepNet. Given a traffic network $G= (V,E)$ and time series $X$ as an example of spatial-temproal datasets. $\bigcirc\!\!\!\!1$ Tokenize and embed $X$ using a customized embedding layer, reprogramming with condensed text prototypes for modality alignment. $\bigcirc\!\!\!\!2$ Incorporate dataset descriptions, task instructions, and statistical characteristics as prompt prefixes to guide input transformation. $\bigcirc\!\!\!\!3$ Leverage a Hypergraph Spatio-Temporal module to model complex spatial dependencies and node-level variations via hierarchical representation learning. $\bigcirc\!\!\!\!4$ Incident matrix: real geographic network, if not, Adaptive Graph or Adaptive HyperGraph is used. By integrating $\bigcirc\!\!\!\!1$$\bigcirc\!\!\!\!2$$\bigcirc\!\!\!\!3$ , STH-SepNet generate the forecasts.
  • Figure 3: Performance comparison of MAE between STH-SepNet trained on different datasets.
  • Figure 4: Analysis of effective order on adaptive hypergraph.
  • Figure 5: Comparison of GPU and time complexity.
  • ...and 2 more figures

Theorems & Definitions (5)

  • Definition 1
  • Theorem 1
  • Definition A.1
  • Definition A.2
  • Definition A.3