OATS: Online Data Augmentation for Time Series Foundation Models

TL;DR

This work tackles the challenge of high-quality data for Time Series Foundation Models (TSFMs) by proposing Online Data Augmentation for Time Series Foundation Models (OATS), a principled, step-aware augmentation framework. OATS uses Time-Series Influence Scores (TSIS) to quantify the value of training samples via data attribution with respect to a small reference set, and then conditions a diffusion-based generator on high-quality guiding signals to synthesize realistic time series data. An explore–exploit mechanism selectively recalculates TSIS to balance augmentation benefit with computational cost, utilizing cached signals to sample high-quality data efficiently. Empirical results across six validation datasets and two TSFM architectures show that OATS consistently improves performance (NLL and MAPE) over static augmentation baselines and regular training, with notable gains on several datasets and robustness to explore–exploit settings. Overall, OATS establishes a practical, scalable approach to dynamically optimize TSFM training data through principled influence assessment and diffusion-based synthesis, enhancing cross-domain generalization and robustness in time-series forecasting tasks.

Abstract

Time Series Foundation Models (TSFMs) are a powerful paradigm for time series analysis and are often enhanced by synthetic data augmentation to improve the training data quality. Existing augmentation methods, however, typically rely on heuristics and static paradigms. Motivated by dynamic data optimization, which shows that the contribution of samples varies across training stages, we propose OATS (Online Data Augmentation for Time Series Foundation Models), a principled strategy that generates synthetic data tailored to different training steps. OATS leverages valuable training samples as principled guiding signals and dynamically generates high-quality synthetic data conditioned on them. We further design a diffusion-based framework to produce realistic time series and introduce an explore-exploit mechanism to balance efficiency and effectiveness. Experiments on TSFMs demonstrate that OATS consistently outperforms regular training and yields substantial performance gains over static data augmentation baselines across six validation datasets and two TSFM architectures. The code is available at the link https://github.com/microsoft/TimeCraft.

Figures (9)

• Figure 1: Architecture of OATS. OATS employs three modules: ① Time-Series Influence Scores (TSIS) create generation guiding signals as high-quality data samples, and ② guide time series synthetic data generation for augmentation. ③ Explore-exploit mechanism comprehensively plans if updating the TSIS or leveraging cached scores.
• Figure 2: The architecture of the denoising diffusion model to generate synthetic data conditioned on constructed generation signals.
• Figure 3: Test loss (NLL) of OATS, TSMixup, Jitter and Regular training for each training step.
• Figure 4: Performance of OATS on different explore-exploit ratio $\epsilon$. (First Row) Test NLL on ETTh1. (Second Row) Test NLL on Electricity. The best performance is labeled by a red circle.
• Figure 5: Comparison between sub-dataset data contribution and data portion.

Theorems & Definitions (2)

• Proposition 1: First-order Taylor approximated influence score
• Lemma 1: Ghost Inner Product