Effective Dataset Distillation for Spatio-Temporal Forecasting with Bi-dimensional Compression

TL;DR

STemDist is proposed, the first dataset distillation method specialized for spatio-temporal time series forecasting and shows empirically that datasets distilled by this method enable model training to be faster, more memory-efficient, and more effective.

Abstract

Spatio-temporal time series are widely used in real-world applications, including traffic prediction and weather forecasting. They are sequences of observations over extensive periods and multiple locations, naturally represented as multidimensional data. Forecasting is a central task in spatio-temporal analysis, and numerous deep learning methods have been developed to address it. However, as dataset sizes and model complexities continue to grow in practice, training deep learning models has become increasingly time- and resource-intensive. A promising solution to this challenge is dataset distillation, which synthesizes compact datasets that can effectively replace the original data for model training. Although successful in various domains, including time series analysis, existing dataset distillation methods compress only one dimension, making them less suitable for spatio-temporal datasets, where both spatial and temporal dimensions jointly contribute to the large data volume. To address this limitation, we propose STemDist, the first dataset distillation method specialized for spatio-temporal time series forecasting. A key idea of our solution is to compress both temporal and spatial dimensions in a balanced manner, reducing training time and memory. We further reduce the distillation cost by performing distillation at the cluster level rather than the individual location level, and we complement this coarse-grained approach with a subset-based granular distillation technique that enhances forecasting performance. On five real-world datasets, we show empirically that, compared to both general and time-series dataset distillation methods, datasets distilled by our STemDist method enable model training (1) faster (up to 6X) (2) more memory-efficient (up to 8X), and (3) more effective (with up to 12% lower prediction error).

Figures (7)

• Figure 1: Synthetic data compressed in both temporal and spatial dimensions in a balanced manner yields significantly lower training time and GPU memory usage compared to datasets of the same size compressed only along the temporal dimension. See Preliminary Experiment \ref{['preliminary_experiment']} for details.
• Figure 2: An example illustrating location clustering, where eight locations in $\mathcal{X}_\mathcal{T}$ are reduced into four locations (cluster centroids) in $\mathcal{X}_\mathcal{C}$. Entries with the same color correspond to the data for the same location. Refer to Example \ref{['cluster:example']} for details.
• Figure 3: An illustrative example showing how the clustered spatio-temporal dataset (which is averaged within each cluster) are partitioned into two location subsets and distilled at the subset level. Entries with the same color correspond to the results for the same location. Refer to Example \ref{['subset:example']} for details.
• Figure 4: Training on synthetic datasets distilled by STemDist is faster and more memory-efficient than training on those distilled by the baselines.
• Figure 5: Scalability of STemDist. Its empirical distillation time grows (sub-)linearly with (a) the number of original time series, (b) the location count in the original dataset, and (c) the feature count in the original dataset.

Theorems & Definitions (8)

• Definition 1: Spatio-temporal Time Series
• Definition 2: Spatio-temporal Time Series Dataset
• Definition 3: Synthetic Dataset
• Example 1
• Example 2
• Theorem 1: Time Complexity of Model Training
• Theorem 2: Space Complexity of Model Training
• Theorem 3: Time Complexity of STemDist