Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive time series forecasting with markovian variance switching

Baptiste Abélès, Joseph de Vilmarest, Olivier Wintemberger

TL;DR

This work tackles adaptive time series forecasting under regime changes by modeling variance switching in a linear Gaussian state-space model with a hidden Markov chain $Z_t$. It reframes SKF as online learning with expert aggregation over different covariance regimes, using a log-likelihood loss and gradient-based optimization for the observation variance, plus a sliding-window variant to sharpen discrimination among experts. The authors introduce Kalman-Hedge for variance selection (KFMH) and demonstrate robustness to misspecification, including a misspecified setting where $\hat{Q}_t=\sum_k p_t(k) Q^{(k)}$ is used, with empirical validation on synthetic data and real electricity load forecasts. The results indicate improved forecast accuracy and resilience compared with standard Kalman-aggregation baselines, highlighting the practical value for adaptive time-series prediction in nonstationary environments.

Abstract

Adaptive time series forecasting is essential for prediction under regime changes. Several classical methods assume linear Gaussian state space model (LGSSM) with variances constant in time. However, there are many real-world processes that cannot be captured by such models. We consider a state-space model with Markov switching variances. Such dynamical systems are usually intractable because of their computational complexity increasing exponentially with time; Variational Bayes (VB) techniques have been applied to this problem. In this paper, we propose a new way of estimating variances based on online learning theory; we adapt expert aggregation methods to learn the variances over time. We apply the proposed method to synthetic data and to the problem of electricity load forecasting. We show that this method is robust to misspecification and outperforms traditional expert aggregation.

Adaptive time series forecasting with markovian variance switching

TL;DR

This work tackles adaptive time series forecasting under regime changes by modeling variance switching in a linear Gaussian state-space model with a hidden Markov chain . It reframes SKF as online learning with expert aggregation over different covariance regimes, using a log-likelihood loss and gradient-based optimization for the observation variance, plus a sliding-window variant to sharpen discrimination among experts. The authors introduce Kalman-Hedge for variance selection (KFMH) and demonstrate robustness to misspecification, including a misspecified setting where is used, with empirical validation on synthetic data and real electricity load forecasts. The results indicate improved forecast accuracy and resilience compared with standard Kalman-aggregation baselines, highlighting the practical value for adaptive time-series prediction in nonstationary environments.

Abstract

Adaptive time series forecasting is essential for prediction under regime changes. Several classical methods assume linear Gaussian state space model (LGSSM) with variances constant in time. However, there are many real-world processes that cannot be captured by such models. We consider a state-space model with Markov switching variances. Such dynamical systems are usually intractable because of their computational complexity increasing exponentially with time; Variational Bayes (VB) techniques have been applied to this problem. In this paper, we propose a new way of estimating variances based on online learning theory; we adapt expert aggregation methods to learn the variances over time. We apply the proposed method to synthetic data and to the problem of electricity load forecasting. We show that this method is robust to misspecification and outperforms traditional expert aggregation.
Paper Structure (23 sections, 8 theorems, 46 equations, 3 figures, 2 tables, 5 algorithms)

This paper contains 23 sections, 8 theorems, 46 equations, 3 figures, 2 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Notations
  4. Exponential-Weight-Average (EWA) algorithm for Kalman Filter
  5. Fixed-Share (FS) algorithm
  6. Methodology
  7. Well-specified setting
  8. Misspecified setting
  9. Choice of loss and optimization over $\sigma$
  10. Sliding Window
  11. Experiments
  12. Experiments on Synthetic Data
  13. Well-Specified (WS) Data with $\sigma_t^2 = \sigma^2$
  14. Data Generation
  15. Misspecified (MS) Data
  16. ...and 8 more sections

Key Result

Theorem 1

(Kalman Filter)

Figures (3)

  • Figure 1: Aggregation of Kalman (left) versus Kalman with aggregation of variances (right).
  • Figure 2: Evolution of the weights $\boldsymbol{p}_t$ for WS (left) and MS Data (right).
  • Figure 3: Evolution of $\sigma_t$ starting from 2 different initial points for WS (left) and MS Data (right).

Theorems & Definitions (13)

  • Theorem 1
  • Definition 1
  • Proposition 1
  • Definition 2
  • Proposition 2
  • Proposition 3
  • proof
  • Proposition 4
  • proof
  • Proposition 5
  • ...and 3 more