Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Modified Bayesian Criterion for Model Selection in Mixed and Hierarchical Frameworks

Diogenes de Jesus Ramirez, Anderson Melchor Hernandez, Isabel Cristina Ramirez, Luis Raúl Pericchi

TL;DR

This paper introduces BIC\_HES, a curvature-aware model selection criterion for mixture and hierarchical models by adding $\log\det(P)$, where $P$ is the Fisher information matrix, to the classical $\text{BIC}$. The approach is motivated by the second-order geometry of the log-likelihood and is supported by Corollary 1, a key theorem linking the criterion to perturbed Bayes factors, and a consistency result showing correct model recovery in large samples. The authors demonstrate the method theoretically and through extensive simulations across linear, mixed, binomial, and Poisson models, showing improved performance in small-sample regimes and when noise variables are present. They also provide explicit computations of $\log\det(P)$ in illustrative hierarchical models and discuss predictive-density based estimation via $\widehat{\mathrm{lpd}}$. The work offers a practically implementable, geometry-informed alternative to traditional information criteria with potential applications in complex data environments.

Abstract

In this work, we propose a modified Bayesian Information Criterion (BIC) specifically designed for mixture models and hierarchical structures. This criterion incorporates the determinant of the Hessian matrix of the log-likelihood function, thereby refining the classical Bayes Factor by accounting for the curvature of the likelihood surface. Such geometric information introduces a more nuanced penalization for model complexity. The proposed approach improves model selection, particularly under small-sample conditions or in the presence of noise variables. Through theoretical derivations and extensive simulation studies-including both linear and linear mixed models-we show that our criterion consistently outperforms traditional methods such as BIC, Akaike Information Criterion (AIC), and related variants. The results suggest that integrating curvature-based information from the likelihood landscape leads to more robust and accurate model discrimination in complex data environments.

A Modified Bayesian Criterion for Model Selection in Mixed and Hierarchical Frameworks

TL;DR

This paper introduces BIC\_HES, a curvature-aware model selection criterion for mixture and hierarchical models by adding , where is the Fisher information matrix, to the classical . The approach is motivated by the second-order geometry of the log-likelihood and is supported by Corollary 1, a key theorem linking the criterion to perturbed Bayes factors, and a consistency result showing correct model recovery in large samples. The authors demonstrate the method theoretically and through extensive simulations across linear, mixed, binomial, and Poisson models, showing improved performance in small-sample regimes and when noise variables are present. They also provide explicit computations of in illustrative hierarchical models and discuss predictive-density based estimation via . The work offers a practically implementable, geometry-informed alternative to traditional information criteria with potential applications in complex data environments.

Abstract

In this work, we propose a modified Bayesian Information Criterion (BIC) specifically designed for mixture models and hierarchical structures. This criterion incorporates the determinant of the Hessian matrix of the log-likelihood function, thereby refining the classical Bayes Factor by accounting for the curvature of the likelihood surface. Such geometric information introduces a more nuanced penalization for model complexity. The proposed approach improves model selection, particularly under small-sample conditions or in the presence of noise variables. Through theoretical derivations and extensive simulation studies-including both linear and linear mixed models-we show that our criterion consistently outperforms traditional methods such as BIC, Akaike Information Criterion (AIC), and related variants. The results suggest that integrating curvature-based information from the likelihood landscape leads to more robust and accurate model discrimination in complex data environments.
Paper Structure (12 sections, 3 theorems, 65 equations, 13 figures)

This paper contains 12 sections, 3 theorems, 65 equations, 13 figures.

Table of Contents

  1. Introduction
  2. Background and Related Works
  3. The Bayes factor
  4. A perturbation of Bayes factor
  5. The proposed criterion
  6. Examples
  7. Simulations
  8. Simulation Study for the Linear Model
  9. Simulations for the Linear Mixed Model
  10. Simulations for Binomial and Poisson Models
  11. Proofs
  12. Conclusions

Key Result

Corollary 3.1

Let $\Theta_{i}$ be a sample set in ${\mathbb R}^{d}$ depending of family of parameters of size $p$ and suppose that it is invariant by rotations ( i.e., for all square matrix $P$ such that $P^{2}=P$, then $P\Theta=\Theta$). Suppose that $\tilde{\theta}_{i}\in \Theta$ is the vector of parameters whi

Figures (13)

  • Figure 1: Information criteria vs sample size
  • Figure 2: Comparison between BIC_HES y BIC.
  • Figure 3: Comparison between BIC_HES y AIC.
  • Figure 4: Information criteria vs number of noise variables
  • Figure 5: Information criteria by number of noise variables and sample size
  • ...and 8 more figures

Theorems & Definitions (6)

  • Corollary 3.1
  • Theorem 3.2
  • Theorem 3.3
  • proof : Proof of Corollary \ref{['1cor']}
  • proof : Proof of Theorem \ref{['betterthm']}
  • proof