DynST: Dynamic Sparse Training for Resource-Constrained Spatio-Temporal Forecasting

TL;DR

DynST tackles the practical challenge of deploying resource-constrained spatio-temporal forecasts by introducing a data-level Dynamic Sparse Training framework. It learns a differentiable sensor-mask over historical regions and employs stream morph, iterative pruning, and drop-regrow cycles to identify and preserve crucial data while discarding irrelevant regions, achieving sparsities of $30\%\sim60\%$ with minimal accuracy loss. The approach is model-agnostic and demonstrates substantial inference speedups across GNN and non-GNN backbones on diverse datasets (WeatherBench, FIT, Taxibj+, EAGLE), including long-horizon predictions. By mapping temporal dynamics into scalable spatial representations, DynST offers industry-level sensor deployment optimization that aligns with real-world constraints and scalability needs.

Abstract

The ever-increasing sensor service, though opening a precious path and providing a deluge of earth system data for deep-learning-oriented earth science, sadly introduce a daunting obstacle to their industrial level deployment. Concretely, earth science systems rely heavily on the extensive deployment of sensors, however, the data collection from sensors is constrained by complex geographical and social factors, making it challenging to achieve comprehensive coverage and uniform deployment. To alleviate the obstacle, traditional approaches to sensor deployment utilize specific algorithms to design and deploy sensors. These methods \textit{dynamically adjust the activation times of sensors to optimize the detection process across each sub-region}. Regrettably, formulating an activation strategy generally based on historical observations and geographic characteristics, which make the methods and resultant models were neither simple nor practical. Worse still, the complex technical design may ultimately lead to a model with weak generalizability. In this paper, we introduce for the first time the concept of spatio-temporal data dynamic sparse training and are committed to adaptively, dynamically filtering important sensor distributions. To our knowledge, this is the \textbf{first} proposal (\textit{termed DynST}) of an \textbf{industry-level} deployment optimization concept at the data level. However, due to the existence of the temporal dimension, pruning of spatio-temporal data may lead to conflicts at different timestamps. To achieve this goal, we employ dynamic merge technology, along with ingenious dimensional mapping to mitigate potential impacts caused by the temporal aspect. During the training process, DynST utilize iterative pruning and sparse training, repeatedly identifying and dynamically removing sensor perception areas that contribute the least to future predictions.

Figures (8)

• Figure 1: Motivation of our proposal.
• Figure 2: Overview of our proposed DynST framework.
• Figure 3: The process of stream morph operator. Each rectangular block and circle node can be interpreted as a sensor recorder.
• Figure 4: An overview of the anticipated JD Technology Platform, we represent the importance of sensors with a gradient from light to dark blue, effectively removing the deployment in the white areas to emphasize this gradation of significance.
• Figure 5: The performance visualization of FIT datasets. We can see that the overall temperature deviation is within 10 degrees Celsius, meeting the requirements of the fire science field. emmons1986needed