Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Consistency of Bayes factors for linear models

Elías Moreno, J. J. Serrano-Pérez, F. Torres-Ruiz

TL;DR

This work analyzes the consistency of Bayes factors for linear models as the number of candidate regressors grows with sample size, comparing intrinsic priors and a spectrum of g-prior mixtures. It provides exact asymptotic results: for $p=O(n^{b})$ with $0\,\le b<1$, intrinsic priors and several g-prior mixtures (including Zellner–Siow, degenerate FS, and Bayarri–style priors) are consistent, while certain priors (notably Liang's original $g$-prior, Cui's CG prior, and broad robust-g-prior subclasses) can be inconsistent; when $p=O(n)$, consistency degrades and explicit inconsistency sets in terms of the limit ratio $r=n/p$ and pseudo-distance $oldsymbol{\delta}$ determine where Bayes factors fail. The paper also assesses finite-sample behavior via posterior model probabilities, showing that some priors yield more decisive discrimination between $M_0$ and $M_p$ than others. Overall, it highlights the critical role of prior choice in model selection for growing model space and provides guidance on priors that maintain consistency under common growth regimes. The results have practical implications for variable selection in regression, especially in high-dimensional or clustered settings.

Abstract

The quality of a Bayes factor crucially depends on the number of regressors, the sample size and the prior on the regression parameters, and hence it has to be established in a case-by-case basis. In this paper we analyze the consistency of a wide class of Bayes factors when the number of potential regressors grows as the sample size grows. We have found that when the number of regressors is finite some classes of priors yield inconsistency, and\ when the potential number of regressors grows at the same rate than the sample size different priors yield different degree of inconsistency. For moderate sample sizes, we evaluate the Bayes factors by comparing the posterior model probability. This gives valuable information to discriminate between the priors for the model parameters commonly used for variable selection.

Consistency of Bayes factors for linear models

TL;DR

This work analyzes the consistency of Bayes factors for linear models as the number of candidate regressors grows with sample size, comparing intrinsic priors and a spectrum of g-prior mixtures. It provides exact asymptotic results: for with , intrinsic priors and several g-prior mixtures (including Zellner–Siow, degenerate FS, and Bayarri–style priors) are consistent, while certain priors (notably Liang's original -prior, Cui's CG prior, and broad robust-g-prior subclasses) can be inconsistent; when , consistency degrades and explicit inconsistency sets in terms of the limit ratio and pseudo-distance determine where Bayes factors fail. The paper also assesses finite-sample behavior via posterior model probabilities, showing that some priors yield more decisive discrimination between and than others. Overall, it highlights the critical role of prior choice in model selection for growing model space and provides guidance on priors that maintain consistency under common growth regimes. The results have practical implications for variable selection in regression, especially in high-dimensional or clustered settings.

Abstract

The quality of a Bayes factor crucially depends on the number of regressors, the sample size and the prior on the regression parameters, and hence it has to be established in a case-by-case basis. In this paper we analyze the consistency of a wide class of Bayes factors when the number of potential regressors grows as the sample size grows. We have found that when the number of regressors is finite some classes of priors yield inconsistency, and\ when the potential number of regressors grows at the same rate than the sample size different priors yield different degree of inconsistency. For moderate sample sizes, we evaluate the Bayes factors by comparing the posterior model probability. This gives valuable information to discriminate between the priors for the model parameters commonly used for variable selection.
Paper Structure (20 sections, 5 theorems, 31 equations, 2 figures)

This paper contains 20 sections, 5 theorems, 31 equations, 2 figures.

Table of Contents

  1. Introduction
  2. The models
  3. Priors for the model parameters and Bayes factors
  4. Intrinsic priors and Bayes factor for heteroscedastic models
  5. Intrinsic prior and Bayes factor for homoscedastic models
  6. Mixtures of the Zellner's $g$-prior and Bayes factors
  7. The Zellner-Siow's mixture of $g$-prior
  8. A degenerate mixtures of $g$-prior
  9. The prior $\pi^{L}(g \,|\, n)$
  10. The prior $\pi ^{CG}(g \,|\, n)$
  11. The prior $\pi^{B}(g \,|\, n,p)$
  12. The robust g-prior class
  13. Asymptotic
  14. The case $\mathbf{b<1}$
  15. The case $\mathbf{b=1}$
  16. ...and 5 more sections

Key Result

Lemma 1

For $p=O(n^{b})$ with $0 \leq b \leq 1$ and the pseudo-distance between a sampling model in $M_{p}$ and $M_{0}$ we have that

Figures (2)

  • Figure 1: Posterior probability of model $M_{0}$ in the set $\{M_{0},M_{p}\}$ as a function of $\mathcal{B}_{p0}$.
  • Figure 2: Inconsistency set of $B_{p0}^{IP}$, top left panel, $B_{p0}^{IPH}$, top right panel, $B_{p0}^{B}$, bottom left panel, and $B_{p0}^{ZS}$, bottom right panel.

Theorems & Definitions (10)

  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Theorem 1
  • proof
  • Corollary 1
  • proof
  • Theorem 2
  • proof