Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Low-rank variational Bayes correction to the Laplace method

Janet van Niekerk, Haavard Rue

TL;DR

The paper introduces Low-Rank Variational Bayes correction (VBC), a hybrid approach that preserves the scalability of the Laplace method while enhancing posterior mean accuracy via a low-rank, propagated mean correction $\pmb\psi_1 = \pmb\psi_0 + \pmb Q_I^{-1}\pmb\lambda$ in reduced dimension $p$. The methodology is implemented as a standalone VBC and as INLA-VBC to improve INLA for latent Gaussian models with hyperparameters, leveraging Gauss–Hermite quadrature and smart gradient techniques to maintain efficiency. Across simulated low-count Poisson data and real-world LGMs (Tokyo rainfall and AML survival with SPDE), VBC and especially INLA-VBC achieve accuracy comparable to MCMC or Laplace with substantially reduced computational cost. This framework offers a flexible, scalable path for accurate approximate Bayesian inference in high-dimensional latent spaces and broad LGMs, with implementation in R-INLA and potential extensions to variance and skewness corrections.

Abstract

Approximate inference methods like the Laplace method, Laplace approximations and variational methods, amongst others, are popular methods when exact inference is not feasible due to the complexity of the model or the abundance of data. In this paper we propose a hybrid approximate method called Low-Rank Variational Bayes correction (VBC), that uses the Laplace method and subsequently a Variational Bayes correction in a lower dimension, to the joint posterior mean. The cost is essentially that of the Laplace method which ensures scalability of the method, in both model complexity and data size. Models with fixed and unknown hyperparameters are considered, for simulated and real examples, for small and large datasets.

Low-rank variational Bayes correction to the Laplace method

TL;DR

The paper introduces Low-Rank Variational Bayes correction (VBC), a hybrid approach that preserves the scalability of the Laplace method while enhancing posterior mean accuracy via a low-rank, propagated mean correction in reduced dimension . The methodology is implemented as a standalone VBC and as INLA-VBC to improve INLA for latent Gaussian models with hyperparameters, leveraging Gauss–Hermite quadrature and smart gradient techniques to maintain efficiency. Across simulated low-count Poisson data and real-world LGMs (Tokyo rainfall and AML survival with SPDE), VBC and especially INLA-VBC achieve accuracy comparable to MCMC or Laplace with substantially reduced computational cost. This framework offers a flexible, scalable path for accurate approximate Bayesian inference in high-dimensional latent spaces and broad LGMs, with implementation in R-INLA and potential extensions to variance and skewness corrections.

Abstract

Approximate inference methods like the Laplace method, Laplace approximations and variational methods, amongst others, are popular methods when exact inference is not feasible due to the complexity of the model or the abundance of data. In this paper we propose a hybrid approximate method called Low-Rank Variational Bayes correction (VBC), that uses the Laplace method and subsequently a Variational Bayes correction in a lower dimension, to the joint posterior mean. The cost is essentially that of the Laplace method which ensures scalability of the method, in both model complexity and data size. Models with fixed and unknown hyperparameters are considered, for simulated and real examples, for small and large datasets.
Paper Structure (18 sections, 47 equations, 6 figures, 3 tables)

This paper contains 18 sections, 47 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Proposal
  3. Low-rank variational correction
  4. Computational aspects
  5. Smart gradient
  6. Expected log-likelihood
  7. Illustrative example - low-count Poisson regression model
  8. Expected log-likelihood
  9. Simulation results
  10. Application to latent Gaussian models
  11. INLA
  12. INLA-VBC
  13. Simulation results
  14. Real data examples
  15. Cyclic second order random walk - small scale
  16. ...and 3 more sections

Figures (6)

  • Figure 1: Poisson counts simulated from \ref{['ex1_eq']} (left) and the marginal posterior of $\beta_0$ (center) and $\beta_1$ (right) from MCMC (blue histogram), HMC (red histogram), the Laplace method (dashed line) and VBC (solid line) based on the prior (broken line) for $n=20$ (top) and $n=100$ (bottom)
  • Figure 2: Poisson counts simulated from \ref{['ex1_eq']} (top left) and the marginal posterior of $\beta_0$ (top right), $\beta_1$ (bottom left) and $\tau$ (bottom right) from the Gaussian strategy (points), Laplace strategy (dashed line), INLA-VBC (solid line) and MCMC
  • Figure 3: Posterior mean of $\pmb\alpha$ (left) (zoomed for the first two months (center)) and the marginal posterior of $\alpha_{339}$ (right) from the Gaussian strategy (points), INLA-VBC (solid blue line), INLA (broken line) and MCMC (solid purple line and histogram)
  • Figure 4: Exact residential locations of patients with AML (dots) and the triangulated mesh for the finite element method estimation of the SPDE of the spatial field
  • Figure 5: Marginal posteriors from the Gaussian strategy (points), INLA-VBC (solid line) and INLA (broken line) for the fixed effects and posterior mean and $95\%$ credible interval for the baseline hazard $h_0(t)$
  • ...and 1 more figures