Singularities in Bayesian Inference: Crucial or Overstated?

TL;DR

The paper introduces Gambel distributions, a unifying folded framework for shrinkage priors in high-dimensional regression, by representing scale mixtures of Exponential Power laws with three-parameter Beta mixtures and showing their Gamma-Beta quotient representation. It connects shrinkage priors to reliability and wealth-distribution theory, enabling new analyses of origin and tail behavior via hazard rates and Lorenz ordering. The authors derive exact density, cdf, and moments for Gambel, establish strong posterior consistency under broad conditions, and develop MCMC schemes for both low- and high-dimensional scenarios, together with practical guidance on hyperparameters. Simulation studies demonstrate that Gambel priors (notably Gambel 1) perform comparably to Horseshoe across varied sparsity levels and data settings, while offering flexible tail and origin behavior through tunable hyperparameters. The work provides a theoretically grounded, computationally viable pathway to selecting shrinkage priors and suggests broader applicability in sparse regression and reliability-inspired analyses.

Abstract

Over the past two decades, shrinkage priors have become increasingly popular, and many proposals can be found in the literature. These priors aim to shrink small effects to zero while maintaining true large effects. Horseshoe-type priors have been particularly successful in various applications, mainly due to their computational advantages. However, there is no clear guidance on choosing the most appropriate prior for a specific setting. In this work, we propose a framework that encompasses a large class of shrinkage distributions, including priors with and without a singularity at zero. By reframing such priors in the context of reliability theory and wealth distributions, we provide insights into the prior parameters and shrinkage properties. The paper's key contributions are based on studying the folded version of such distributions, which we refer to as the Gambel distribution. The Gambel can be rewritten as the ratio between a Generalised Gamma and a Generalised Beta of the second kind. This representation allows us to gain insights into the behaviours near the origin and along the tails, compute measures to compare their distributional properties, derive consistency results, devise MCMC schemes for posterior inference and ultimately provide guidance on the choice of the hyperparameters.

Figures (5)

• Figure 1: Density plots of the scale mixtures in Theorem \ref{['th:density']} for different hyperparameters (light solid line) versus the Normal density with same variance (dark thin line).
• Figure 2: Hazard rates of the folded Gambel distribution for various hyperparameters.
• Figure 3: Illustration of Lorenz Curve and Gini Index: The thick line depicts a Lorenz curve. The Gini index is calculated as the ratio of the area of the shaded gray region ($A$) to the total area under the perfect equality line ($A+B$).
• Figure 4: Lorenz curves of scale mixtures for different hyperparameters. Dashed lines denote the improper prior distribution of perfect equality.
• Figure 5: Prior densities (left) and their shrinkage profiles (right). The shrinkage profile of the Horseshoe prior cannot be seen as it is superimposed by the Gambel 1 prior.

Theorems & Definitions (23)

• Theorem 1
• Corollary 1
• Proposition 1
• Theorem 2
• Proposition 2
• Proposition 3
• Remark 1
• Theorem 3
• Theorem 4
• Proposition 4