Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Variational Inference for Count Response Semiparametric Regression: A Convex Solution

Virginia Murru, Matt P. Wand

TL;DR

The paper presents a convex, closed-form variational inference framework for count-response semiparametric regression that leverages Polya-Gamma augmentation of the Negative Binomial likelihood and a structured mean-field approach with a discrete shape-parameter prior. By conditioning on a finite set of $\kappa$ atoms and exploiting convex updates, the method achieves fast, stable fitting suitable for real-time, online inference, while maintaining competitive accuracy to MCMC benchmarks. The authors additionally introduce online streaming adaptations that maintain only low-dimensional sufficient statistics, enabling purely online processing without storing all data. Empirical results on simulated data and pollen-counts data demonstrate favorable speed, reasonable accuracy, and practical applicability to count data with random effects and spline components.

Abstract

We develop a version of variational inference for Bayesian count response regression-type models that possesses attractive attributes such as convexity and closed form updates. The convex solution aspect entails numerically stable fitting algorithms, whilst the closed form aspect makes the methodology fast and easy to implement. The essence of the approach is the use of Pólya-Gamma augmentation of a Negative Binomial likelihood, a finite-valued prior on the shape parameter and the structured mean field variational Bayes paradigm. The approach applies to general count response situations. For concreteness, we focus on generalized linear mixed models within the semiparametric regression class of models. Real-time fitting is also described.

Variational Inference for Count Response Semiparametric Regression: A Convex Solution

TL;DR

The paper presents a convex, closed-form variational inference framework for count-response semiparametric regression that leverages Polya-Gamma augmentation of the Negative Binomial likelihood and a structured mean-field approach with a discrete shape-parameter prior. By conditioning on a finite set of atoms and exploiting convex updates, the method achieves fast, stable fitting suitable for real-time, online inference, while maintaining competitive accuracy to MCMC benchmarks. The authors additionally introduce online streaming adaptations that maintain only low-dimensional sufficient statistics, enabling purely online processing without storing all data. Empirical results on simulated data and pollen-counts data demonstrate favorable speed, reasonable accuracy, and practical applicability to count data with random effects and spline components.

Abstract

We develop a version of variational inference for Bayesian count response regression-type models that possesses attractive attributes such as convexity and closed form updates. The convex solution aspect entails numerically stable fitting algorithms, whilst the closed form aspect makes the methodology fast and easy to implement. The essence of the approach is the use of Pólya-Gamma augmentation of a Negative Binomial likelihood, a finite-valued prior on the shape parameter and the structured mean field variational Bayes paradigm. The approach applies to general count response situations. For concreteness, we focus on generalized linear mixed models within the semiparametric regression class of models. Real-time fitting is also described.
Paper Structure (25 sections, 98 equations, 6 figures, 3 algorithms)

This paper contains 25 sections, 98 equations, 6 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Notation
  3. Model Description
  4. Extension to Covariance Matrix Parameters for Random Effects
  5. Variational Inference Scheme
  6. Streamlined Variational Inference Alternatives
  7. Real-Time Count Response Semiparametric Regression
  8. The Step of Algorithm \ref{['alg:ONLalgo']} Involving the Reduced Atom Set ${\mathcal{K}}_n$
  9. Numerical Results
  10. Simulated Data Assessment of Algorithm \ref{['alg:BCHalgo']}
  11. Assessment of Accuracy
  12. Assessment of Speed
  13. Simulated Data Assessment of Algorithm \ref{['alg:ONLalgo']}
  14. Application to Pollen Counts Data
  15. Conclusion
  16. ...and 10 more sections

Figures (6)

  • Figure 1: Directed acyclic graph representation of model (\ref{['eq:negBinModel']}) with incorporation of the $\boldsymbol{a}=(a_1,\ldots,a_r)$ auxiliary variables as in (\ref{['eq:HCtoIG']}). The $\boldsymbol{y}$ node is shaded to indicate that it contains observed data.
  • Figure 2: Boxplots of accuracy scores, as defined by (\ref{['eq:accDefn']}), for the Algorithm \ref{['alg:BCHalgo']} simulation study. The $f(Q_k,Q_{k'})$ notation is defined by (\ref{['eq:fdefn']}) and subsequent text.
  • Figure 3: Illustrations of the accuracy of the structured mean field variational Bayes (MFVB) posterior density functions and probability mass function approximations obtained from Algorithm \ref{['alg:BCHalgo']}. In each panel, the MFVB approximate density functions or probability mass function for a quantity of interest is compared with its Markov chain Monte Carlo (MCMC) counterpart. The percentage is the accuracy score according to (\ref{['eq:accDefn']}). The vertical lines indicate true values according to the simulation set-up.
  • Figure 4: Some illustrative comparisons for $\kappa_{\hbox{\tiny true}}=5$ between the real-time Negative Binomial nonparametric regression estimates based on Algorithm \ref{['alg:ONLalgo']} with the batch counterparts based on Algorithm \ref{['alg:BCHalgo']}. The solid curves correspond to posterior means. The dashed curves correspond to pointwise approximate 95% credible intervals. The scatterplots correspond to the current regression data.
  • Figure 5: Comparison of posterior density function and probability mass function approximations based on structured mean field variational Bayes (MFVB) and Markov chain Monte Carlo (MCMC) for four of the parameters in model (\ref{['eq:ragweedModel']}).
  • ...and 1 more figures