Deep Temporal Deaggregation: Large-Scale Spatio-Temporal Generative Models

TL;DR

This work tackles privacy-constrained mobility time-series by developing TDDPM, a transformer-based diffusion model that can generate long spatio-temporal sequences. It achieves scalability by conditioning the diffusion process on spatial occupancy priors $p(l)$, enabling deaggregation to individual trajectories via $p(x|l)$ and supporting what-if analyses in changing environments. A hierarchical sampling strategy over sub-regions, using local, rotation- and translation-invariant heatmaps, scales city-sized data while preserving fidelity and distributional proportionality. The approach is validated on a diverse benchmark, showing competitive unconditional performance, substantial efficiency gains, and strong out-of-distribution generalization, including scenario modeling for road network changes.

Abstract

Many of today's data is time-series data originating from various sources, such as sensors, transaction systems, or production systems. Major challenges with such data include privacy and business sensitivity. Generative time-series models have the potential to overcome these problems, allowing representative synthetic data, such as people's movement in cities, to be shared openly and be used to the benefit of society at large. However, contemporary approaches are limited to prohibitively short sequences and small scales. Aside from major memory limitations, the models generate less accurate and less representative samples the longer the sequences are. This issue is further exacerbated by the lack of a comprehensive and accessible benchmark. Furthermore, a common need in practical applications is what-if analysis and dynamic adaptation to data distribution changes, for usage in decision making and to manage a changing world: What if this road is temporarily blocked or another road is added? The focus of this paper is on mobility data, such as people's movement in cities, requiring all these issues to be addressed. To this end, we propose a transformer-based diffusion model, TDDPM, for time-series which outperforms and scales substantially better than state-of-the-art. This is evaluated in a new comprehensive benchmark across several sequence lengths, standard datasets, and evaluation measures. We also demonstrate how the model can be conditioned on a prior over spatial occupancy frequency information, allowing the model to generate mobility data for previously unseen environments and for hypothetical scenarios where the underlying road network and its usage changes. This is evaluated by training on mobility data from part of a city. Then, using only aggregate spatial information as prior, we demonstrate out-of-distribution generalization to the unobserved remainder of the city.

Figures (7)

• Figure 1: TDDPM is trained on real 2D trajectories (left) to generate synthetic trajectories (right), conditioned on how likely it should be for the population of synthetic trajectories to occupy space on the 2D plane. The latter is represented as a discrete distribution over occupancy frequency, i.e., the marginal distribution over the trajectory probability distribution if integrating out time. This yields both high-fidelity in-distribution generalization (top) and out-of-distribution generalization (bottom), the latter when conditioning on a marginal distribution not part of the training data (dashed rectangle).
• Figure 2: Overview of the architecture. In the unconditional part, each time point of the noisy trajectory is converted into a separate token with positional embedding vaswani2017attention used to embed its values and the time point, as well as a learned vector representing its type. The denoising step token encodes the denoising step, the step is encoding using positional encoding and then concatenated with a type vector. The transformer can also optionally take a marginal distribution to guide the denoising process to generate samples with particular properties, improving in-distribution performance as well as enabling generalization to previously unobserved areas. The marginal distribution is split into tokens, taking inspiration from ViT dosovitskiy2020image, concatenated with a learned type vector and the corresponding position using positional embedding.
• Figure 3: Interpolation experiment. The model has trained on this region is tasked to reconstruct it from the heatmaps. Top left: Data from GeoLife used for training. Top right: heatmap of training data and areas used for creating query heatmaps for sampling the model, Bottom left: synthetic trajectories. Bottom right: heatmap of the synthetic data.
• Figure 4: Generalization experiment. The model is trained on the lower left quadrant and used to generate data on the remaining geographical area. Top left: Data from GeoLife, the lower quandrant of which used for training. Top right: heatmap of training data, Bottom left: synthetic trajectories. Bottom right: heatmap of the synthetic data.
• Figure 5: t-SNE analysis for sequence length 64. Blank figures are unsuccessful experiments (Table \ref{['table:result-TimeFID']}).