Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ANT: Adaptive Noise Schedule for Time Series Diffusion Models

Seunghan Lee, Kibok Lee, Taeyoung Park

TL;DR

This work proposes Adaptive Noise schedule for Time series diffusion models (ANT), which automatically predetermines proper noise schedules for given TS datasets based on their statistics representing non-stationarity, and validate the effectiveness of the method across various tasks, including TS forecasting, refinement, and generation.

Abstract

Advances in diffusion models for generative artificial intelligence have recently propagated to the time series (TS) domain, demonstrating state-of-the-art performance on various tasks. However, prior works on TS diffusion models often borrow the framework of existing works proposed in other domains without considering the characteristics of TS data, leading to suboptimal performance. In this work, we propose Adaptive Noise schedule for Time series diffusion models (ANT), which automatically predetermines proper noise schedules for given TS datasets based on their statistics representing non-stationarity. Our intuition is that an optimal noise schedule should satisfy the following desiderata: 1) It linearly reduces the non-stationarity of TS data so that all diffusion steps are equally meaningful, 2) the data is corrupted to the random noise at the final step, and 3) the number of steps is sufficiently large. The proposed method is practical for use in that it eliminates the necessity of finding the optimal noise schedule with a small additional cost to compute the statistics for given datasets, which can be done offline before training. We validate the effectiveness of our method across various tasks, including TS forecasting, refinement, and generation, on datasets from diverse domains. Code is available at this repository: https://github.com/seunghan96/ANT.

ANT: Adaptive Noise Schedule for Time Series Diffusion Models

TL;DR

This work proposes Adaptive Noise schedule for Time series diffusion models (ANT), which automatically predetermines proper noise schedules for given TS datasets based on their statistics representing non-stationarity, and validate the effectiveness of the method across various tasks, including TS forecasting, refinement, and generation.

Abstract

Advances in diffusion models for generative artificial intelligence have recently propagated to the time series (TS) domain, demonstrating state-of-the-art performance on various tasks. However, prior works on TS diffusion models often borrow the framework of existing works proposed in other domains without considering the characteristics of TS data, leading to suboptimal performance. In this work, we propose Adaptive Noise schedule for Time series diffusion models (ANT), which automatically predetermines proper noise schedules for given TS datasets based on their statistics representing non-stationarity. Our intuition is that an optimal noise schedule should satisfy the following desiderata: 1) It linearly reduces the non-stationarity of TS data so that all diffusion steps are equally meaningful, 2) the data is corrupted to the random noise at the final step, and 3) the number of steps is sufficiently large. The proposed method is practical for use in that it eliminates the necessity of finding the optimal noise schedule with a small additional cost to compute the statistics for given datasets, which can be done offline before training. We validate the effectiveness of our method across various tasks, including TS forecasting, refinement, and generation, on datasets from diverse domains. Code is available at this repository: https://github.com/seunghan96/ANT.

Paper Structure

This paper contains 35 sections, 17 equations, 26 figures, 18 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Works
  3. Methodology
  4. ANT: Adaptive Noise Schedule for TS Diffusion Models
  5. Diffusion Step Embedding for TS Diffusion Models
  6. Robustness of Non-linear Schedules
  7. Experiments
  8. Time Series Forecasting
  9. Time Series Refinement
  10. Time Series Generation
  11. Analysis
  12. Conclusion
  13. Dataset Description
  14. Experimental Settings
  15. Baseline Methods
  16. ...and 20 more sections

Figures (26)

  • Figure 1: Performance gain by ANT.
  • Figure 2: Overall framework of ANT. (a) shows that a base schedule (w/o ANT) abruptly corrupts TS at the earlier diffusion step, while a schedule proposed by ANT gradually corrupts it until the final step. (b) visualizes the non-stationarity curves of both schedules and their discrepancy from a linear line, with the schedule that gradually decreases the non-stationarity (w/ ANT) showing lower discrepancy. (c) shows that better performance is achieved as the curves get closer to a linear line.
  • Figure 3: Non-stationarity curves of various schedules.
  • Figure 4: Desiderata of noise schedules.
  • Figure 5: Proxy task classification & t-SNE visualization.
  • ...and 21 more figures