Approximate Gibbs Sampler for Efficient Inference of Hierarchical Bayesian Models for Grouped Count Data

TL;DR

This work addresses scalable inference for hierarchical Bayesian Poisson regression models on grouped count data by introducing an Approximate Gibbs Sampler (AGS) that exploits a Gaussian approximation to the Poisson likelihood via a log-gamma Gaussian surrogate. The key innovation is closed-form approximate posteriors for the regression coefficients, enabling fast Gibbs updates while preserving accuracy on large datasets. Empirical results on synthetic and real data show substantial speedups—often exceeding an order of magnitude—against NUTS, with comparable predictive performance; however, performance degrades when many zeros or very small counts are present, where NUTS can outperform AGS. Overall, AGS offers a practical, scalable option for time-sensitive applications requiring HBPRMs, with clear guidance on when to prefer exact MCMC methods for accuracy in low-count regimes.

Abstract

Hierarchical Bayesian Poisson regression models (HBPRMs) provide a flexible modeling approach of the relationship between predictors and count response variables. The applications of HBPRMs to large-scale datasets require efficient inference algorithms due to the high computational cost of inferring many model parameters based on random sampling. Although Markov Chain Monte Carlo (MCMC) algorithms have been widely used for Bayesian inference, sampling using this class of algorithms is time-consuming for applications with large-scale data and time-sensitive decision-making, partially due to the non-conjugacy of many models. To overcome this limitation, this research develops an approximate Gibbs sampler (AGS) to efficiently learn the HBPRMs while maintaining the inference accuracy. In the proposed sampler, the data likelihood is approximated with Gaussian distribution such that the conditional posterior of the coefficients has a closed-form solution. Numerical experiments using real and synthetic datasets with small and large counts demonstrate the superior performance of AGS in comparison to the state-of-the-art sampling algorithm, especially for large datasets.

Figures (2)

• Figure 1: Quality of the Gaussian approximation (dashed blue line) to the true distribution (solid red line) for different values of $y$: (a) $y$=1; (b) $y$=3; (c) $y$=5; (d) $y$=10; (e) $y$=20. (f) Values of KS distance (solid red line) between the approximate distribution and true distribution and values of the absolute error in the mean (dashed blue line) of the approximate distribution as the value of $y$ increases.
• Figure 2: Scalability of NUTS and AGS on (a) real datasets and (b) synthetic datasets. The size of a dataset, i.e., the total number of count data points, can be calculated by $\sum_{j=1}^{J} n_j$.