Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Geometric Ergodicity of Gibbs Algorithms for a Normal Model With a Global-Local Shrinkage Prior

Yasuyuki Hamura

TL;DR

This paper establishes geometric ergodicity for Gibbs samplers in a normal linear regression model with global-local shrinkage priors, first under the Horseshoe with a three-parameter beta global prior and easy global-parameter updates, and then for a broad class of priors using a reparameterized likelihood with rejection sampling. It proves ergodicity under mild conditions, including finite moments or bounded support, and introduces practical, rejection-based samplers with an improved two-stage/partially collapsed scheme to reduce autocorrelation. A simulation study compares the proposed and existing JOB methods, highlighting efficiency in sampling ${ au^2}$ and ${oldsymbol{eta}}$, with performance depending on the prior and dimensionality ${p}$. The results provide both theoretical guarantees and practical algorithms for robust Bayesian inference with global-local shrinkage priors in high-dimensional settings.

Abstract

We consider Gibbs samplers for a normal linear regression model with a global-local shrinkage prior and show that they produce geometrically ergodic Markov chains. First, under the horseshoe local prior and a three-parameter beta global prior under some assumptions, we prove geometric ergodicity for a Gibbs algorithm in which it is relatively easy to update the global shrinkage parameter. Second, we consider a more general class of global-local shrinkage priors. Under milder conditions, geometric ergodicity is proved for two- and three-stage Gibbs samplers based on rejection sampling. We also construct a practical rejection sampling method in the horseshoe case. Finally, a simulation study is performed to compare proposed and existing methods.

Geometric Ergodicity of Gibbs Algorithms for a Normal Model With a Global-Local Shrinkage Prior

TL;DR

This paper establishes geometric ergodicity for Gibbs samplers in a normal linear regression model with global-local shrinkage priors, first under the Horseshoe with a three-parameter beta global prior and easy global-parameter updates, and then for a broad class of priors using a reparameterized likelihood with rejection sampling. It proves ergodicity under mild conditions, including finite moments or bounded support, and introduces practical, rejection-based samplers with an improved two-stage/partially collapsed scheme to reduce autocorrelation. A simulation study compares the proposed and existing JOB methods, highlighting efficiency in sampling and , with performance depending on the prior and dimensionality . The results provide both theoretical guarantees and practical algorithms for robust Bayesian inference with global-local shrinkage priors in high-dimensional settings.

Abstract

We consider Gibbs samplers for a normal linear regression model with a global-local shrinkage prior and show that they produce geometrically ergodic Markov chains. First, under the horseshoe local prior and a three-parameter beta global prior under some assumptions, we prove geometric ergodicity for a Gibbs algorithm in which it is relatively easy to update the global shrinkage parameter. Second, we consider a more general class of global-local shrinkage priors. Under milder conditions, geometric ergodicity is proved for two- and three-stage Gibbs samplers based on rejection sampling. We also construct a practical rejection sampling method in the horseshoe case. Finally, a simulation study is performed to compare proposed and existing methods.

Paper Structure

This paper contains 40 sections, 11 theorems, 231 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. The Horseshoe Case
  3. The model and an algorithm
  4. Geometric ergodicity
  5. The General Case
  6. The model and algorithms
  7. Geometric ergodicity
  8. An improved sampler
  9. A Simulation Study
  10. Discussion
  11. Proof of Proposition \ref{['prp:normal-cauchy']}
  12. The case of $s > 1$
  13. The case of $s > 1$ and $r > 1$
  14. The case of $s > 1$ and $r \le 1$
  15. The case of $s \le 1$
  16. ...and 25 more sections

Key Result

Theorem 2.1

Figures (2)

  • Figure 1: Boxplots of effective sample sizes of ${\beta} _1 , \dots , {\beta} _{10}$ for the methods of Section \ref{['sec:horseshoe']} (new1), Section \ref{['subsec:improved']} (new2), bkp2022 (mjob), and job2020 (ujob) based on five datasets with $(n, p) = (100, 25)$.
  • Figure 2: Boxplots of effective sample sizes of ${\beta} _1 , \dots , {\beta} _{10}$ for the methods of Section \ref{['sec:horseshoe']} (new1), Section \ref{['subsec:improved']} (new2), bkp2022 (mjob), and job2020 (ujob) based on five datasets with $(n, p) = (100, 75)$.

Theorems & Definitions (21)

  • Theorem 2.1
  • Proposition 3.1
  • Theorem 3.1
  • Proposition 3.2
  • Remark S2.1
  • Remark S2.2
  • Remark S2.3
  • Remark S2.4
  • Remark S2.5
  • Remark S2.6
  • ...and 11 more