Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Forging Time Series with Language: A Large Language Model Approach to Synthetic Data Generation

Cécile Rousseau, Tobia Boschi, Giandomenico Cornacchia, Dhaval Salwala, Alessandra Pascale, Juan Bernabe Moreno

TL;DR

SDForger tackles the challenge of generating high-quality multivariate time series in data-scarce settings by decoupling representation learning from generation. It converts time-series into compact tabular embeddings via Functional Principal Components or FastICA, encodes these embeddings as textual prompts, and fine-tunes lightweight autoregressive LLMs to generate new embedding rows, which are decoded back into time-series data. The approach yields synthetic sequences that preserve statistical properties and temporal dynamics, improving downstream forecasting without sacrificing diversity, and it scales efficiently with long windows due to the embedding-based representation. Extensive experiments across diverse domains show competitive or superior similarity and utility metrics compared with strong baselines, and results remain robust across ablations and LLM backbones. By enabling textual conditioning and multimodal extensions, SDForger offers a flexible pathway to integrating time series with natural language and other modalities in practical applications.

Abstract

SDForger is a flexible and efficient framework for generating high-quality multivariate time series using LLMs. Leveraging a compact data representation, SDForger provides synthetic time series generation from a few samples and low-computation fine-tuning of any autoregressive LLM. Specifically, the framework transforms univariate and multivariate signals into tabular embeddings, which are then encoded into text and used to fine-tune the LLM. At inference, new textual embeddings are sampled and decoded into synthetic time series that retain the original data's statistical properties and temporal dynamics. Across a diverse range of datasets, SDForger outperforms existing generative models in many scenarios, both in similarity-based evaluations and downstream forecasting tasks. By enabling textual conditioning in the generation process, SDForger paves the way for multimodal modeling and the streamlined integration of time series with textual information. The model is open-sourced at https://github.com/IBM/fms-dgt/tree/main/fms_dgt/public/databuilders/time_series.

Forging Time Series with Language: A Large Language Model Approach to Synthetic Data Generation

TL;DR

SDForger tackles the challenge of generating high-quality multivariate time series in data-scarce settings by decoupling representation learning from generation. It converts time-series into compact tabular embeddings via Functional Principal Components or FastICA, encodes these embeddings as textual prompts, and fine-tunes lightweight autoregressive LLMs to generate new embedding rows, which are decoded back into time-series data. The approach yields synthetic sequences that preserve statistical properties and temporal dynamics, improving downstream forecasting without sacrificing diversity, and it scales efficiently with long windows due to the embedding-based representation. Extensive experiments across diverse domains show competitive or superior similarity and utility metrics compared with strong baselines, and results remain robust across ablations and LLM backbones. By enabling textual conditioning and multimodal extensions, SDForger offers a flexible pathway to integrating time series with natural language and other modalities in practical applications.

Abstract

SDForger is a flexible and efficient framework for generating high-quality multivariate time series using LLMs. Leveraging a compact data representation, SDForger provides synthetic time series generation from a few samples and low-computation fine-tuning of any autoregressive LLM. Specifically, the framework transforms univariate and multivariate signals into tabular embeddings, which are then encoded into text and used to fine-tune the LLM. At inference, new textual embeddings are sampled and decoded into synthetic time series that retain the original data's statistical properties and temporal dynamics. Across a diverse range of datasets, SDForger outperforms existing generative models in many scenarios, both in similarity-based evaluations and downstream forecasting tasks. By enabling textual conditioning in the generation process, SDForger paves the way for multimodal modeling and the streamlined integration of time series with textual information. The model is open-sourced at https://github.com/IBM/fms-dgt/tree/main/fms_dgt/public/databuilders/time_series.

Paper Structure

This paper contains 56 sections, 11 equations, 3 figures, 17 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Time-series generation
  4. Foundation models and LLMs for time series
  5. Specialized architectures for time-series generation
  6. Methodology
  7. Notation
  8. Periodicity-aware segmentation
  9. From time series to tabular data
  10. Generation of tabular data
  11. From embeddings table to text
  12. Large language model finetuning and inference
  13. Fine-tuning
  14. Retrieve embedding from text
  15. In-generation filtering and stopping criterion
  16. ...and 41 more sections

Figures (3)

  • Figure 1: SDForger pipeline. Overview of the SDForger generation process. The example illustrates a setting with $I=3$ input segments, $C=2$ channels, $k_1 = k_2 = 3$ components, and $\tilde{I} = 2$ generated samples. The model performs periodicity-aware segmentation, extracts embeddings, and embed them into text. An LLM is then fine-tuned to generate embedding sequences, which are finally decoded to reconstruct synthetic time series.
  • Figure 2: Text-Conditioned Generation with SDForger. Visualization of 10 original (grey) and synthetic samples per channel from the bikesharing data. Synthetic data is generated using conditional prompts: "Condition: data is cnt (blue)", "Condition: data is hum (pink)", and "Condition: data is temp (orange)".
  • Figure A.1: Periodicity-aware segmentation.

Theorems & Definitions (2)

  • Definition 1: Textual encoder
  • Definition 2: Textual inference