DynaConF: Dynamic Forecasting of Non-Stationary Time Series

TL;DR

We address non-stationary conditional distributions in time-series forecasting by decoupling the stationary, time-invariant part from time-varying dynamics. DynaConF uses a dynamic control variable $\boldsymbol{\phi}_t = \boldsymbol{\chi}_t + \boldsymbol{b}_{\phi}$, where $\boldsymbol{\chi}_t$ evolves via a Bernoulli-random-walk to capture continuous and abrupt changes, and a deep encoder forms a context $\boldsymbol{h}_t$ whose per-dimension Gaussian observations have means and variances modulated by $\boldsymbol{\phi}_t$. Inference relies on a variational lower bound with a tractable $q(\boldsymbol{\chi}_{B:T})$ and Rao-Blackwellized particle filtering for online adaptation, enabling scalable forecasting for high-dimensional series. Empirically, DynaConF achieves improved adaptation to non-stationary changes and competitive or superior performance across synthetic and real-world datasets compared with state-of-the-art baselines. This framework offers robust, online-friendly forecasting under distribution shifts, with potential for extensions to richer observation models and more scalable posteriors.

Abstract

Deep learning has shown impressive results in a variety of time series forecasting tasks, where modeling the conditional distribution of the future given the past is the essence. However, when this conditional distribution is non-stationary, it poses challenges for these models to learn consistently and to predict accurately. In this work, we propose a new method to model non-stationary conditional distributions over time by clearly decoupling stationary conditional distribution modeling from non-stationary dynamics modeling. Our method is based on a Bayesian dynamic model that can adapt to conditional distribution changes and a deep conditional distribution model that handles multivariate time series using a factorized output space. Our experimental results on synthetic and real-world datasets show that our model can adapt to non-stationary time series better than state-of-the-art deep learning solutions.

Figures (4)

• Figure 1: Overview: Decoupled Model. DynaConF decouples the stationary (time-invariant) modeling (left) and the non-stationary (time-variant) modeling (right) of the conditional distribution $p(\boldsymbol{y}_t|\boldsymbol{y}_{t-1:t-B})$ across time $t$ (middle). The dashed arrows indicate the information flow at time $T-1$ when predicting $\boldsymbol{y}_T$, while the solid arrows indicate the information flow at time $T$ when predicting $\boldsymbol{y}_{T+1}$.
• Figure 2: Architecture. DynaConF is built on the principle of a clean decoupling of stationary conditional distribution modeling (red) and non-stationary dynamics modeling (blue). We predict the parameters ($\boldsymbol{\theta}$) of the conditional distribution (green) by aggregating time-invariant local context ($\boldsymbol{z}$) and modulating this context with time-variant global dynamics ($\boldsymbol{\phi}$) driven by a random walk ($\boldsymbol{\chi}$) with Bernoulli restarts ($\boldsymbol{\pi}$).
• Figure 3: Qualitative Results. We show one dimension of $\boldsymbol{\phi}_t$ of DynaConF--PP inferred with particle filters at test time on (a) AR(1)--Flip, (b) AR(1)--Sin, and (c) AR(1)--Dynamic. The red dashed lines show the parameter $w_t$ in the generative processes varying over time. The blue curves and bands are the medians and 90% confidence intervals of the posterior. Note that because the encoder in our model is non-linear, the encoding $\boldsymbol{z}_t$ that is combined with $\boldsymbol{\phi}_t$ is not the same as $y_{t-1}$, so the inferred $\boldsymbol{\phi}_t$may not match the sign/scale of$w_t$, but we expect it to change gradually or suddenly according to $w_t$.
• Figure 4: Qualitative Results on Electricity Dataset. We show the predictions of StatiConF (a--f) vs DynaConF (g--l) on one dimension of electricity dataset, where there appears to be a change in the distribution at test time.