Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Extension of the Unified Skew-Normal Family of Distributions and Application to Bayesian Binary Regression

Paolo Onorati, Brunero Liseo

TL;DR

A new class of distributions, the Perturbed Unified Skew Normal (pSUN), which generalizes the SUN class is introduced, and it is shown that the new class is conjugate to any binary regression model, provided that the link function may be expressed as a scale mixture of Gaussian densities.

Abstract

We consider the Bayesian binary regression model and we introduce a new class of distributions, the Perturbed Unified Skew-Normal (pSUN, henceforth), which generalizes the Unified Skew-Normal (SUN) class. We show that the new class is conjugate to any binary regression model, provided that the link function may be expressed as a scale mixture of Gaussian CDFs. We discuss in detail the popular logit case, and we show that, when a logistic regression model is combined with a Gaussian prior, posterior summaries such as cumulants and normalizing constants can easily be obtained through the use of an importance sampling approach, opening the way to straightforward variable selection procedures. For more general prior distributions, the proposed methodology is based on a simple Gibbs sampler algorithm. We also claim that, in the p>n case, our proposal presents better performances - both in terms of mixing and accuracy - compared to the existing methods.

An Extension of the Unified Skew-Normal Family of Distributions and Application to Bayesian Binary Regression

TL;DR

A new class of distributions, the Perturbed Unified Skew Normal (pSUN), which generalizes the SUN class is introduced, and it is shown that the new class is conjugate to any binary regression model, provided that the link function may be expressed as a scale mixture of Gaussian densities.

Abstract

We consider the Bayesian binary regression model and we introduce a new class of distributions, the Perturbed Unified Skew-Normal (pSUN, henceforth), which generalizes the Unified Skew-Normal (SUN) class. We show that the new class is conjugate to any binary regression model, provided that the link function may be expressed as a scale mixture of Gaussian CDFs. We discuss in detail the popular logit case, and we show that, when a logistic regression model is combined with a Gaussian prior, posterior summaries such as cumulants and normalizing constants can easily be obtained through the use of an importance sampling approach, opening the way to straightforward variable selection procedures. For more general prior distributions, the proposed methodology is based on a simple Gibbs sampler algorithm. We also claim that, in the p>n case, our proposal presents better performances - both in terms of mixing and accuracy - compared to the existing methods.
Paper Structure (16 sections, 6 theorems, 39 equations, 3 figures, 9 tables)

This paper contains 16 sections, 6 theorems, 39 equations, 3 figures, 9 tables.

Table of Contents

  1. Introduction
  2. The Perturbed SUN Distribution
  3. Sampling from a pSUN distribution
  4. Bayesian Linear Symmetric Binary Regression
  5. Bayesian Logistic Regression
  6. Gibbs Algorithm for Logistic Regression.
  7. Importance Sampling for Logistic Regression with a Gaussian Prior
  8. Examples
  9. Simulation Study: Coverage Analysis
  10. Simulation Study: Polya-Gamma vs pSUN with Small Sample Size
  11. Simulation Study: a Logit Model with Unbalanced Data
  12. Simulation Study: a Probit Model with Sparse Parameters
  13. A Real Dataset: Parameters Estimation for Cancer SAGE
  14. A Real Dataset: Model and Variable Selection for Lung Cancer
  15. Conclusion
  16. ...and 1 more sections

Key Result

Theorem 1

If $\beta\sim \mathrm{pSUN}_{p,m}\left( Q_V, \Theta, A, b, Q_W, \Omega, \xi \right)$ then the density function and the MGF of $\beta$ can be written as

Figures (3)

  • Figure 1: Polya-Gamma vs $\mathrm{pSUN}$ with small sample size: Mean of ACF for the Intercept, $\mathrm{pSUN}$-Gibbs (continuous line) vs PG (dashed line) vs UPG (dotted line). Left: Gaussian Prior. Center: Laplacit Prior. Right: Dirichlet-Laplace Prior.
  • Figure 2: Logit model with Unbalanced Data: ACF Comparison, $\mathrm{pSUN}$-Gibbs (continuous line) vs PG (dashed line) vs UPG (dotted line). Left: $n = 50$. Center: $n = 200$. Right: $n = 1000$.
  • Figure 3: Cancer SAGE Example, Posterior Means of the 516 $\beta$ Coefficients Plus The Intercept $\beta_1$ Using the $\mathrm{pSUN}$-Gibbs Algorithm. Top: Logit. Bottom: Probit. Left: Gaussian Prior. Center: Laplacit Prior. Right: Dirichlet-Laplace Prior.

Theorems & Definitions (7)

  • Definition 1
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Theorem 5
  • Theorem 6