Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TimeLDM: Latent Diffusion Model for Unconditional Time Series Generation

Jian Qian, Bingyu Xie, Biao Wan, Minhao Li, Miao Sun, Patrick Yin Chiang

TL;DR

TimeLDM tackles unconditional time series generation by transferring the modeling task from the data space to a latent space via a beta-VAE and applying a latent diffusion model to latent representations. This two stage approach yields high fidelity synthetic time series across simulated and real datasets, surpassing state-of-the-art baselines on multiple metrics. The work demonstrates robustness to varying time series lengths and highlights the benefits of latent diffusion for time series generation. The proposed method introduces a strong, generalizable baseline for generating informative time series tokens for agents learning in physical AI.

Abstract

Time series generation is a crucial research topic in the area of decision-making systems, which can be particularly important in domains like autonomous driving, healthcare, and, notably, robotics. Recent approaches focus on learning in the data space to model time series information. However, the data space often contains limited observations and noisy features. In this paper, we propose TimeLDM, a novel latent diffusion model for high-quality time series generation. TimeLDM is composed of a variational autoencoder that encodes time series into an informative and smoothed latent content and a latent diffusion model operating in the latent space to generate latent information. We evaluate the ability of our method to generate synthetic time series with simulated and real-world datasets and benchmark the performance against existing state-of-the-art methods. Qualitatively and quantitatively, we find that the proposed TimeLDM persistently delivers high-quality generated time series. For example, TimeLDM achieves new state-of-the-art results on the simulated benchmarks and an average improvement of 55% in Discriminative score with all benchmarks. Further studies demonstrate that our method yields more robust outcomes across various lengths of time series data generation. Especially, for the Context-FID score and Discriminative score, TimeLDM realizes significant improvements of 80% and 50%, respectively. The code will be released after publication.

TimeLDM: Latent Diffusion Model for Unconditional Time Series Generation

TL;DR

TimeLDM tackles unconditional time series generation by transferring the modeling task from the data space to a latent space via a beta-VAE and applying a latent diffusion model to latent representations. This two stage approach yields high fidelity synthetic time series across simulated and real datasets, surpassing state-of-the-art baselines on multiple metrics. The work demonstrates robustness to varying time series lengths and highlights the benefits of latent diffusion for time series generation. The proposed method introduces a strong, generalizable baseline for generating informative time series tokens for agents learning in physical AI.

Abstract

Time series generation is a crucial research topic in the area of decision-making systems, which can be particularly important in domains like autonomous driving, healthcare, and, notably, robotics. Recent approaches focus on learning in the data space to model time series information. However, the data space often contains limited observations and noisy features. In this paper, we propose TimeLDM, a novel latent diffusion model for high-quality time series generation. TimeLDM is composed of a variational autoencoder that encodes time series into an informative and smoothed latent content and a latent diffusion model operating in the latent space to generate latent information. We evaluate the ability of our method to generate synthetic time series with simulated and real-world datasets and benchmark the performance against existing state-of-the-art methods. Qualitatively and quantitatively, we find that the proposed TimeLDM persistently delivers high-quality generated time series. For example, TimeLDM achieves new state-of-the-art results on the simulated benchmarks and an average improvement of 55% in Discriminative score with all benchmarks. Further studies demonstrate that our method yields more robust outcomes across various lengths of time series data generation. Especially, for the Context-FID score and Discriminative score, TimeLDM realizes significant improvements of 80% and 50%, respectively. The code will be released after publication.
Paper Structure (12 sections, 7 equations, 6 figures, 6 tables, 2 algorithms)

This paper contains 12 sections, 7 equations, 6 figures, 6 tables, 2 algorithms.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORKS
  3. METHOD
  4. Problem Statement
  5. Time Series Autoencoding
  6. Latent Diffusion Model
  7. Training and Sampling
  8. EXPERIMENTS
  9. Experimental Setups
  10. Unconditional Time Series Generation
  11. Ablation Study
  12. CONCLUSIONS

Figures (6)

  • Figure 1: t-SNE visualization on the stocks dataset, TimeLDM shows better overlap between the generated data and original data than TimeVAE.
  • Figure 2: Structure of our proposed TimeLDM. (a) shows the components of TimeLDM, consisting of the transformer encoder, reparameterization, diffusion process, reverse process, and transformer decoder. (b) shows the details of the transformer encoder and reparameterization. (c) shows the architecture of the latent diffusion model. (d) shows the details of the transformer decoder.
  • Figure 3: Visualizations of the simulated MuJoCo dataset, synthesized by TimeLDM, Diffusion-TS and TimeVAE.
  • Figure 4: Visualizations of the real-world ETTh dataset, synthesized by TimeLDM, Diffusion-TS, and TimeVAE.
  • Figure 5: Examples of generating time series from the fMRI dataset. Our approach yields the closest results to the original training data.
  • ...and 1 more figures