Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Recurrent Interpolants for Probabilistic Time Series Prediction

Yu Chen, Marin Biloš, Sarthak Mittal, Wei Deng, Kashif Rasul, Anderson Schneider

TL;DR

This work addresses probabilistic forecasting for high-dimensional time series by marrying recurrent networks' efficiency with diffusion-based generative modeling through stochastic interpolants. It extends the stochastic interpolant framework to conditional generation with extra features and develops a conditional SI module that leverages an RNN-encoded history as guidance for predicting future distributions. Empirically, the approach is evaluated on synthetic and real multivariate datasets, showing competitive or superior performance to DDPM, SGM, and FM baselines in most settings, while illustrating the benefits of conditioning and importance sampling for stable training. The proposed method promises scalable, high-fidelity probabilistic forecasting by fusing sequence modeling with flexible, condition-aware diffusion dynamics, with potential impact on domains requiring reliable uncertainty quantification in time series.

Abstract

Sequential models like recurrent neural networks and transformers have become standard for probabilistic multivariate time series forecasting across various domains. Despite their strengths, they struggle with capturing high-dimensional distributions and cross-feature dependencies. Recent work explores generative approaches using diffusion or flow-based models, extending to time series imputation and forecasting. However, scalability remains a challenge. This work proposes a novel method combining recurrent neural networks' efficiency with diffusion models' probabilistic modeling, based on stochastic interpolants and conditional generation with control features, offering insights for future developments in this dynamic field.

Recurrent Interpolants for Probabilistic Time Series Prediction

TL;DR

This work addresses probabilistic forecasting for high-dimensional time series by marrying recurrent networks' efficiency with diffusion-based generative modeling through stochastic interpolants. It extends the stochastic interpolant framework to conditional generation with extra features and develops a conditional SI module that leverages an RNN-encoded history as guidance for predicting future distributions. Empirically, the approach is evaluated on synthetic and real multivariate datasets, showing competitive or superior performance to DDPM, SGM, and FM baselines in most settings, while illustrating the benefits of conditioning and importance sampling for stable training. The proposed method promises scalable, high-fidelity probabilistic forecasting by fusing sequence modeling with flexible, condition-aware diffusion dynamics, with potential impact on domains requiring reliable uncertainty quantification in time series.

Abstract

Sequential models like recurrent neural networks and transformers have become standard for probabilistic multivariate time series forecasting across various domains. Despite their strengths, they struggle with capturing high-dimensional distributions and cross-feature dependencies. Recent work explores generative approaches using diffusion or flow-based models, extending to time series imputation and forecasting. However, scalability remains a challenge. This work proposes a novel method combining recurrent neural networks' efficiency with diffusion models' probabilistic modeling, based on stochastic interpolants and conditional generation with control features, offering insights for future developments in this dynamic field.
Paper Structure (24 sections, 2 theorems, 31 equations, 6 figures, 8 tables, 2 algorithms)

This paper contains 24 sections, 2 theorems, 31 equations, 6 figures, 8 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Denoising Diffusion Probabilistic Model (DDPM)
  4. Score-based Generative Model (SGM)
  5. Flow Matching (FM)
  6. Stochastic Interpolants (SI)
  7. Conditional generation with extra features
  8. Stochastic interpolants for time series prediction
  9. Experiments
  10. Synthetic datasets
  11. Multivariate probabilistic forecasting
  12. Models.
  13. Setup.
  14. Results.
  15. Conclusions
  16. ...and 9 more sections

Key Result

Theorem 1

(Extension of Stochastic Interpolants to Arbitrary Joint Distributions). Let $\rho_{01}$ be the joint distribution $(\mathbf{x}^0, \mathbf{x}^1) \sim \rho_{01}$ and let the stochastic interpolant be where $\alpha_0=\beta_1=1$, $\alpha_1=\beta_0=\gamma_0 = \gamma_1 = 0$, and $\alpha_s^2+\beta_s^2+\gamma_s^2>0$ for all $s\in[0,1]$. We define $\rho_s$ to be the noise-dependent density of $\mathbf{x}

Figures (6)

  • Figure 1: $\alpha(\cdot)$, $\beta(\cdot)$, and $\gamma(\cdot)$, the schedules of stochastic interpolants.
  • Figure 2: Stochastic interpolants for time series prediction using forward SDE in equation \ref{['eq:si_forward_sde_cond']}.
  • Figure 3: Examples of model generated samples for synthetic two-dimensional ($D=2$) datasets.
  • Figure 4: Forecast paths for SI on Solar dataset showing median prediction, 50th and 90th confidence intervals calculated from model samples, on $6 \;/\; 137$ variate dimensions.
  • Figure 5: Time-Grad rasul2021autoregressive model for conditional time series prediction as a comparison.
  • ...and 1 more figures

Theorems & Definitions (4)

  • Theorem 1
  • proof : Proof of Theorem \ref{['SI_theorem']}
  • Theorem 2
  • proof