Information criteria for astrophysical model selection

TL;DR

The paper reviews information criteria for astrophysical model selection, focusing on Bayesian evidence, AIC/BIC, and the Deviance Information Criterion (DIC). It argues that DIC provides a practical bridge that leverages posterior samples to address parameter degeneracy, and applies these criteria to WMAP3 cosmology. The findings show substantial divergence between information-theoretic (AIC/DIC) and Bayesian approaches, with DIC resembling AIC in practice, and with the full Bayes evidence and BIC yielding different rankings depending on priors. This work underscores how prior assumptions and data constraints shape model selection in cosmology and motivates using DIC for pre-existing posterior analyses.

Abstract

Model selection is the problem of distinguishing competing models, perhaps featuring different numbers of parameters. The statistics literature contains two distinct sets of tools, those based on information theory such as the Akaike Information Criterion (AIC), and those on Bayesian inference such as the Bayesian evidence and Bayesian Information Criterion (BIC). The Deviance Information Criterion combines ideas from both heritages; it is readily computed from Monte Carlo posterior samples and, unlike the AIC and BIC, allows for parameter degeneracy. I describe the properties of the information criteria, and as an example compute them from WMAP3 data for several cosmological models. I find that at present the information theory and Bayesian approaches give significantly different conclusions from that data.