BayesBlend: Easy Model Blending using Pseudo-Bayesian Model Averaging, Stacking and Hierarchical Stacking in Python

TL;DR

The BayesBlend Python package, which provides a user-friendly programming interface to estimate weights and blend multiple (Bayesian) models' predictive distributions, and implements pseudo-Bayesian model averaging, stacking and, uniquely, hierarchical Bayesian stacking to estimate model weights.

Abstract

Averaging predictions from multiple competing inferential models frequently outperforms predictions from any single model, providing that models are optimally weighted to maximize predictive performance. This is particularly the case in so-called $\mathcal{M}$-open settings where the true model is not in the set of candidate models, and may be neither mathematically reifiable nor known precisely. This practice of model averaging has a rich history in statistics and machine learning, and there are currently a number of methods to estimate the weights for constructing model-averaged predictive distributions. Nonetheless, there are few existing software packages that can estimate model weights from the full variety of methods available, and none that blend model predictions into a coherent predictive distribution according to the estimated weights. In this paper, we introduce the BayesBlend Python package, which provides a user-friendly programming interface to estimate weights and blend multiple (Bayesian) models' predictive distributions. BayesBlend implements pseudo-Bayesian model averaging, stacking and, uniquely, hierarchical Bayesian stacking to estimate model weights. We demonstrate the usage of BayesBlend with examples of insurance loss modeling.

Figures (5)

• Figure 1: Blended posterior distributions and ELPD values from the deveopment example. In the first column, black crosses on the blended posterior densities indicate the real data. In the second column, ELPD point estimates and 1 standard deviation ranges are shown by open and filled circles, denoting candidate model and blended ELPD estimates, respectively, and the horizontal line segments.
• Figure 2: Model weights and log posterior predictive density (LPD) values from the loss development example. Model weights are constant across development years for every model except the hierarchical stacking model, which included development year as a discrete covariate. The left y-axis illustrates the weight values for each of the blending models. The right y-axis denotes the mean (open circles) and 1 standard deviation across programs (vertical line segments) LPD values.
• Figure 3: Ten randomly selected programs of 50 used for the forecasting example. Each program has 10 close-to-ultimate incurred loss ratios (circles), developed to 10 years. Each candidate model is fit to all 50 programs through year 9 and predictions are validated at year 10 (red points). Leave-future-out cross-validation is used to predict years 6 through 9 (blue points) to create a test-set of data to train the blending models.
• Figure 4: Blended posterior distributions and ELPD values from the forecasting example. In the first column, black crosses on the blended posterior densities indicate the real data. In the second column, ELPD point estimates and 1 standard deviation ranges are shown by open and filled circles, denoting candidate model and blended ELPD estimates, respectively, and the horizontal line segments.
• Figure 5: Model weights and log posterior predictive density (LPD) values from the forecasting model. Model weights are constant across leave-future-out (LFO) accident years for every model except the hierarchical stacking model, which included accident year as a discrete covariate. The left y-axis illustrates the weight values for each of the blending models. The right y-axis denotes the mean (open circles) and 1 standard deviation across programs (vertical line segments) LPD values.