Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Objective Model Prior Probabilities in Variable Selection

James Berger, Gonzalo García-Donato, Elías Moreno, Luis Pericchi

Abstract

For many years it was routine to use equal model prior probabilities in Bayesian model uncertainty analysis. At least twenty years ago it became clear that this was problematic, leading to support of much too large models in the increasingly huge model spaces being considered in genomics and other fields. A popular replacement was to adopt a suggestion of Harold Jeffreys for the variable selection problem in which a total of $k$ possible variables are being considered for inclusion in the model: give the collection of all models containing $d$ variables ($d = 0, . . . , k$) prior probability $1/(k + 1)$ and then divide this prior probability equally among the models in the collection. Many other choices of model prior probabilities that impose severe parsimony have also been introduced. We begin by reviewing the problems with using equal model prior probabilities and then discuss some serious problems with the Jeffreys choice. Finally, we introduce and study a number of objective alternative choices of model prior probabilities, from both numerical and theoretical perspectives.

Objective Model Prior Probabilities in Variable Selection

Abstract

For many years it was routine to use equal model prior probabilities in Bayesian model uncertainty analysis. At least twenty years ago it became clear that this was problematic, leading to support of much too large models in the increasingly huge model spaces being considered in genomics and other fields. A popular replacement was to adopt a suggestion of Harold Jeffreys for the variable selection problem in which a total of possible variables are being considered for inclusion in the model: give the collection of all models containing variables () prior probability and then divide this prior probability equally among the models in the collection. Many other choices of model prior probabilities that impose severe parsimony have also been introduced. We begin by reviewing the problems with using equal model prior probabilities and then discuss some serious problems with the Jeffreys choice. Finally, we introduce and study a number of objective alternative choices of model prior probabilities, from both numerical and theoretical perspectives.
Paper Structure (23 sections, 1 theorem, 33 equations, 2 figures, 1 table)

This paper contains 23 sections, 1 theorem, 33 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction and motivation
  2. Variable selection in the General linear model
  3. More parsimonious prior probabilities
  4. Harmonic prior probabilities
  5. CMG prior probabilities
  6. Half-$p$ Jeffreys prior probabilities
  7. Half-$k$ Jeffreys prior probabilities
  8. Beta and hierarchical Beta prior probabilities
  9. Overly parsimonious choices from the perspective of objectivity
  10. Numerical comparisons of the prior probabilities
  11. Graphs of the priors
  12. A discriminating example
  13. Philosophical and theoretical comparisons
  14. Philosophical issues
  15. When $k=1$
  16. ...and 8 more sections

Key Result

Lemma 1

Using (Bgamma), consider $B_{\gamma}(SSE)$ as a function of $SSE$ for fixed $n, k_0$, $SSE_0$ and $|\gamma|$. If $SSE_1\le SSE_2$ then $B_{\gamma}(SSE_1)\ge B_{\gamma}(SSE_2)$.

Figures (2)

  • Figure 1: For $k=49$, Half-$p$ (black); Jeffreys (red), Uniform (Green), Half-$k$ (blue); CGM (Cyan); Hierarchical (magenta); Beta(1,2) (gray); Harmonic (yellow).
  • Figure 2: For the obesity dataset, $k=47$, the log of ratio of the posterior probability of the most probable model of each dimension to the null model for different priors. Half-$p$ (black); Jeffreys (red), Uniform (Green), Half-$k$ (blue); CGM (Cyan); Hierarchical (magenta); Beta(1,2) (gray); Harmonic (yellow).

Theorems & Definitions (2)

  • Lemma 1
  • proof