Non-stationary Diffusion For Probabilistic Time Series Forecasting

TL;DR

This work tackles the challenge of non-stationary uncertainty in probabilistic time-series forecasting by integrating the Location-Scale Noise Model (LSNM) into denoising diffusion probabilistic models (DDPMs). NsDiff introduces an uncertainty-aware noise schedule and leverages pretrained mean and variance estimators to model time-varying endpoint distributions, enabling adaptive uncertainty representation. Empirical results across nine real-world datasets and synthetic scenarios demonstrate state-of-the-art performance in both accuracy and uncertainty calibration, especially under high non-stationarity. The approach offers a flexible, model-agnostic framework that improves probabilistic forecasting by explicitly accounting for changing variance over time.

Abstract

Due to the dynamics of underlying physics and external influences, the uncertainty of time series often varies over time. However, existing Denoising Diffusion Probabilistic Models (DDPMs) often fail to capture this non-stationary nature, constrained by their constant variance assumption from the additive noise model (ANM). In this paper, we innovatively utilize the Location-Scale Noise Model (LSNM) to relax the fixed uncertainty assumption of ANM. A diffusion-based probabilistic forecasting framework, termed Non-stationary Diffusion (NsDiff), is designed based on LSNM that is capable of modeling the changing pattern of uncertainty. Specifically, NsDiff combines a denoising diffusion-based conditional generative model with a pre-trained conditional mean and variance estimator, enabling adaptive endpoint distribution modeling. Furthermore, we propose an uncertainty-aware noise schedule, which dynamically adjusts the noise levels to accurately reflect the data uncertainty at each step and integrates the time-varying variances into the diffusion process. Extensive experiments conducted on nine real-world and synthetic datasets demonstrate the superior performance of NsDiff compared to existing approaches. Code is available at https://github.com/wwy155/NsDiff.

Figures (5)

• Figure 1: A figure illustrates DDPMs with different endpoints trained to estimate the number of influenza-like disease patients weekly. We plot the endpoint distributions and prediction intervals of $\mathcal{N}(0, \mathbf{I})$ (Top), $\mathcal{N}(f(\mathbf{X}), \mathbf{I})$ (Middle), and $\mathcal{N}(f(\mathbf{X}), g(\mathbf{X}))$ (Bottom) on the left and right, respectively. The red dashed line indicates the division of the training and test dataset.
• Figure 2: The outline of NsDiff. It integrates a LSNM-based endpoint and an uncertainty-aware noise schedule. During the training phase, a noise and variance estimator, $\xi_\theta$, is optimized to approximate the reverse process distribution. To generate a sample, during inference, it samples from LSNM endpoint and use the estimated reverse distribution to iteratively denoise and generate the final prediction.
• Figure 3: The 95% prediction intervals of a ETTh1 sample, the black line is the true values, the red area represents the prediction interval.
• Figure 4: The estimated variance and ground truth in linear variance dataset, the variance is estimated using 100 samples. The red dashed line indicates the split of training and extended test sets.
• Figure 5: The 95% prediction intervals comparison with other models.