Time-Transformer: Integrating Local and Global Features for Better Time Series Generation (Extended Version)

TL;DR

Time-Transformer AAE targets the core challenge of time-series generation by learning both local patterns and global dependencies. It integrates a Time-Transformer module into an adversarial autoencoder decoder, enabling layer-wise parallel processing of local (TCN) and global (Transformer) features with bidirectional cross-attention for fusion, formalized as $\mathcal{F}(\mathcal{P})=\mathcal{I}(\mathcal{L},\mathcal{G})$. Empirically, it delivers state-of-the-art or near-state-of-the-art performance across six diverse datasets, including long sequences and synthetic mixtures, and demonstrates clear benefits in data augmentation for small and imbalanced datasets. Ablation studies confirm the superiority of the parallel design over sequential combinations. The approach offers practical impact for generating high-quality synthetic time series in applications ranging from medical data augmentation to forecasting pipelines, while noting limitations and future extensions such as conditional generation and incomplete data handling.

Abstract

Generating time series data is a promising approach to address data deficiency problems. However, it is also challenging due to the complex temporal properties of time series data, including local correlations as well as global dependencies. Most existing generative models have failed to effectively learn both the local and global properties of time series data. To address this open problem, we propose a novel time series generative model named 'Time-Transformer AAE', which consists of an adversarial autoencoder (AAE) and a newly designed architecture named 'Time-Transformer' within the decoder. The Time-Transformer first simultaneously learns local and global features in a layer-wise parallel design, combining the abilities of Temporal Convolutional Networks and Transformer in extracting local features and global dependencies respectively. Second, a bidirectional cross attention is proposed to provide complementary guidance across the two branches and achieve proper fusion between local and global features. Experimental results demonstrate that our model can outperform existing state-of-the-art models in 5 out of 6 datasets, specifically on those with data containing both global and local properties. Furthermore, we highlight our model's advantage on handling this kind of data via an artificial dataset. Finally, we show our model's ability to address a real-world problem: data augmentation to support learning with small datasets and imbalanced datasets.

Figures (7)

• Figure 1: Time-Transformer AAE
• Figure 2: Detailed design of the Time-Transformer block
• Figure 3: t-SNE visualisations of models (columns) on 3 datasets (rows). The blue dots represent original data and the red dots represent synthetic data.
• Figure 4: Mixture, Global and Local time series
• Figure 5: Results of Time-Trans(green), RTSGAN(blue), and TimeVAE(orange). Lower are better.