A Symmetric Random Scan Collapsed Gibbs Sampler for Fully Bayesian Variable Selection with Spike-and-Slab Priors

TL;DR

This work addresses scalable Bayesian variable selection in ultrahigh dimensions by introducing a symmetric random-scan collapsed Gibbs sampler for spike-and-slab models with a Dirac spike and Laplace-type slab. It combines collapsed marginal likelihoods, on-the-fly Gram computations, and data-informed proposal weights to update only a small subset of coordinates while maintaining the exact posterior, aided by a hierarchical Beta-Bernoulli sparsity prior. A key contribution is the posterior-mean-size rule for adaptive model selection, along with explicit tuning guidance based on the signal-to-null correlation ratio, demonstrated via extensive simulations and a riboflavin-genomics application. The approach offers substantial computational savings in time and memory without sacrificing inferential validity, enabling reliable variable selection in problems with $p$ up to $10^5$ and beyond, with strong practical impact for genomics and related fields.

Abstract

We introduce a symmetric random scan Gibbs sampler for scalable Bayesian variable selection that eliminates storage of the full cross-product matrix by computing required quantities on-the-fly. Data-informed proposal weights, constructed from marginal correlations, concentrate sampling effort on promising candidates while a uniform mixing component ensures theoretical validity. We provide explicit guidance for selecting tuning parameters based on the ratio of signal to null correlations, ensuring adequate posterior exploration. The posterior-mean-size selection rule provides an adaptive alternative to the median probability model that automatically calibrates to the effective signal density without requiring an arbitrary threshold. In simulations with one hundred thousand predictors, the method achieves sensitivity of 1.000 and precision above 0.76. Application to a genomic dataset studying riboflavin production in Bacillus subtilis identifies six genes, all validated by previous studies using alternative methods. The underlying model combines a Dirac spike-and-slab prior with Laplace-type shrinkage: the Dirac spike enforces exact sparsity by setting inactive coefficients to precisely zero, while the Laplace-type slab provides adaptive regularization for active coefficients through a local-global scale mixture.