A unifying framework for generalised Bayesian online learning in non-stationary environments

TL;DR

The paper introduces BONE, a unifying framework for probabilistic online learning in non-stationary environments, built on three modelling choices and two inference algorithms. By casting online prediction in a hierarchical two-level state-space model and allowing a flexible auxiliary variable to drive non-stationarity, BONE encompasses a wide range of existing methods and enables new hybrids. It formalizes the expected posterior predictive and provides concrete instantiations, including a novel RL[1]-OUPR* variant that handles both gradual and abrupt changes, along with a JAX-based open-source library. Across diverse tasks—prequential forecasting, online continual learning, segmentation, and contextual bandits—the framework demonstrates superior or competitive performance and offers a coherent lens to compare methods from different subfields.

Abstract

We propose a unifying framework for methods that perform probabilistic online learning in non-stationary environments. We call the framework BONE, which stands for generalised (B)ayesian (O)nline learning in (N)on-stationary (E)nvironments. BONE provides a common structure to tackle a variety of problems, including online continual learning, prequential forecasting, and contextual bandits. The framework requires specifying three modelling choices: (i) a model for measurements (e.g., a neural network), (ii) an auxiliary process to model non-stationarity (e.g., the time since the last changepoint), and (iii) a conditional prior over model parameters (e.g., a multivariate Gaussian). The framework also requires two algorithmic choices, which we use to carry out approximate inference under this framework: (i) an algorithm to estimate beliefs (posterior distribution) about the model parameters given the auxiliary variable, and (ii) an algorithm to estimate beliefs about the auxiliary variable. We show how the modularity of our framework allows for many existing methods to be reinterpreted as instances of BONE, and it allows us to propose new methods. We compare experimentally existing methods with our proposed new method on several datasets, providing insights into the situations that make each method more suitable for a specific task. We provide a Jax open source library to facilitate the adoption of this framework.

Figures (16)

• Figure 1: Overview of BONE methods grouped by the qualitative nature of the auxiliary variable and the conditional prior. See Table \ref{['tab:related-bone-methods']} for a detailed breakdown of these methods.
• Figure 2: Two-levelled hierarchical state-space model (SSM) with known dynamics, motivating our BONE framework Solid arrows indicate required dependencies, while dashed arrows represent optional dependencies. Rectangles denote exogenous variables, and circles represent random variables. Observed elements are shaded in gray. The left shift in ${\bm{x}}_t$ represents that features are observed before observing ${\bm{y}}_t$.
• Figure 3: The top panel shows the target variable (electricity consumption) from March 1 2020 to March 12 2020. The bottom panel shows the twelve-hour rolling relative absolute error of predictions for the same time window. The dotted black line corresponds to March 7 2020, when Covid lockdown began.
• Figure 4: One day ahead electricity forecasting results for Figure \ref{['fig:day-ahead-plot']}. The dotted black line corresponds to March 7 2020.
• Figure 5: One day ahead electricity forecasting results for together with the target variable on the left y-axis, and the value for runlength (RL) on the right y-axis. We see that after the 7 March changepoint, the runlength monotonically increases, indicating a stationary regime.