Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Going NUTS with ADVI: Exploring various Bayesian Inference techniques with Facebook Prophet

Jovan Krajevski, Biljana Tojtovska Ribarski

TL;DR

This work reimplements Facebook Prophet in PyMC to enable exploration of multiple Bayesian inference techniques beyond Prophet's built-in MAP and NUTS-based MCMC. By evaluating full MCMC (MH, NUTS, DMZ), MAP, and VI (ADVI and FR-ADVI) on a univariate time-series forecasting task, the authors assess convergence diagnostics, forecasting accuracy, and computational efficiency. Key findings show that NUTS converges reliably with favorable ESS and autocorrelation properties, whereas MH/DMZ struggle; VI methods offer substantial speedups with competitive metrics but differing posterior uncertainty representations, particularly FR-ADVI’s tendency to miscalibrate uncertainty. The work highlights trade-offs among accuracy, uncertainty quantification, and computation, and points to GPU acceleration and VI optimization as promising avenues for scalable Bayesian time-series inference in Prophet-like models.

Abstract

Since its introduction, Facebook Prophet has attracted positive attention from both classical statisticians and the Bayesian statistics community. The model provides two built-in inference methods: maximum a posteriori estimation using the L-BFGS-B algorithm, and Markov Chain Monte Carlo (MCMC) sampling via the No-U-Turn Sampler (NUTS). While exploring various time-series forecasting problems using Bayesian inference with Prophet, we encountered limitations stemming from the inability to apply alternative inference techniques beyond those provided by default. Additionally, the fluent API design of Facebook Prophet proved insufficiently flexible for implementing our custom modeling ideas. To address these shortcomings, we developed a complete reimplementation of the Prophet model in PyMC, which enables us to extend the base model and evaluate and compare multiple Bayesian inference methods. In this paper, we present our PyMC-based implementation and analyze in detail the implementation of different Bayesian inference techniques. We consider full MCMC techniques, MAP estimation and Variational inference techniques on a time-series forecasting problem. We discuss in details the sampling approach, convergence diagnostics, forecasting metrics as well as their computational efficiency and detect possible issues which will be addressed in our future work.

Going NUTS with ADVI: Exploring various Bayesian Inference techniques with Facebook Prophet

TL;DR

This work reimplements Facebook Prophet in PyMC to enable exploration of multiple Bayesian inference techniques beyond Prophet's built-in MAP and NUTS-based MCMC. By evaluating full MCMC (MH, NUTS, DMZ), MAP, and VI (ADVI and FR-ADVI) on a univariate time-series forecasting task, the authors assess convergence diagnostics, forecasting accuracy, and computational efficiency. Key findings show that NUTS converges reliably with favorable ESS and autocorrelation properties, whereas MH/DMZ struggle; VI methods offer substantial speedups with competitive metrics but differing posterior uncertainty representations, particularly FR-ADVI’s tendency to miscalibrate uncertainty. The work highlights trade-offs among accuracy, uncertainty quantification, and computation, and points to GPU acceleration and VI optimization as promising avenues for scalable Bayesian time-series inference in Prophet-like models.

Abstract

Since its introduction, Facebook Prophet has attracted positive attention from both classical statisticians and the Bayesian statistics community. The model provides two built-in inference methods: maximum a posteriori estimation using the L-BFGS-B algorithm, and Markov Chain Monte Carlo (MCMC) sampling via the No-U-Turn Sampler (NUTS). While exploring various time-series forecasting problems using Bayesian inference with Prophet, we encountered limitations stemming from the inability to apply alternative inference techniques beyond those provided by default. Additionally, the fluent API design of Facebook Prophet proved insufficiently flexible for implementing our custom modeling ideas. To address these shortcomings, we developed a complete reimplementation of the Prophet model in PyMC, which enables us to extend the base model and evaluate and compare multiple Bayesian inference methods. In this paper, we present our PyMC-based implementation and analyze in detail the implementation of different Bayesian inference techniques. We consider full MCMC techniques, MAP estimation and Variational inference techniques on a time-series forecasting problem. We discuss in details the sampling approach, convergence diagnostics, forecasting metrics as well as their computational efficiency and detect possible issues which will be addressed in our future work.
Paper Structure (22 sections, 1 equation, 5 figures, 8 tables)

This paper contains 22 sections, 1 equation, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Reimplementing Facebook Prophet in PyMC
  3. Overview of the model
  4. Intuitive API
  5. Full MCMC sampling
  6. MAP estimate
  7. Variational Inference
  8. Methodology
  9. Dataset
  10. Model specification
  11. Sampling, Convergence Diagnostics and Forecasting Metrics
  12. Results
  13. MCMC convergence
  14. R-hat
  15. Efective Sample Size
  16. ...and 7 more sections

Figures (5)

  • Figure 1: Autocorrelation plot from Metropolis-Hastings for the slope parameter $k$ in the Linear Trend component
  • Figure 2: Autocorrelation plot from NUTS for the slope parameter $k$ in the Linear Trend component
  • Figure 3: Negative ELBO through iterations
  • Figure 4: NUTS vs ADVI and FullRank ADVI for the slope parameter $k$ in the Linear Trend component
  • Figure 5: NUTS vs MH vs DMZ for the slope parameter $k$ in the Linear Trend component