Likelihood Level Adapted Estimation of Marginal Likelihood for Bayesian Model Selection

TL;DR

The paper tackles the computational bottleneck of Bayesian model selection by recasting the marginal likelihood integral into a one-dimensional form via a probability integral transform and evaluating it with adaptive quadrature over iso-likelihood levels. It introduces three likelihood-level adapted estimators—LLA-IS, LLA-SS, and LLA-MCMC—that generate samples at successive likelihood thresholds using importance sampling, stratified sampling, and MCMC, respectively, to efficiently compute the evidence. Through four numerical examples, including a high-dimensional Karhunen–Loève expansion, the study shows that LLA-SS excels in low-dimensional problems, while LLA-MCMC robustly scales to high-dimensional uncertainty; all three methods outperform traditional nested sampling and generic Monte Carlo in accuracy and efficiency. The results demonstrate practical improvements for Bayesian model selection in complex, multivariate settings and highlight the trade-offs among parallelizability, scalability, and multi-modality handling. This approach offers a flexible, efficient framework for model comparison in computational mechanics and related fields, with potential for parallel implementation and multi-physics extensions.

Abstract

In computational mechanics, multiple models are often present to describe a physical system. While Bayesian model selection is a helpful tool to compare these models using measurement data, it requires the computationally expensive estimation of a multidimensional integral -- known as the marginal likelihood or as the model evidence (\textit{i.e.}, the probability of observing the measured data given the model). This study presents efficient approaches for estimating this marginal likelihood by transforming it into a one-dimensional integral that is subsequently evaluated using a quadrature rule at multiple adaptively-chosen iso-likelihood contour levels. Three different algorithms are proposed to estimate the probability mass at each adapted likelihood level using samples from importance sampling, stratified sampling, and Markov chain Monte Carlo sampling, respectively. The proposed approach is illustrated through four numerical examples. The first example validates the algorithms against a known exact marginal likelihood. The second example uses an 11-story building subjected to an earthquake excitation with an uncertain hysteretic base isolation layer with two models to describe the isolation layer behavior. The third example considers flow past a cylinder when the inlet velocity is uncertain. Based on these examples, the method with stratified sampling is by far the most accurate and efficient method for complex model behavior in low dimension. In the fourth example, the proposed approach is applied to heat conduction in an inhomogeneous plate with uncertain thermal conductivity modeled through a 100 degree-of-freedom Karhunen-Loève expansion. The results indicate that MultiNest cannot efficiently handle the high-dimensional parameter space, whereas the proposed MCMC-based method more accurately and efficiently explores the parameter space.

Figures (9)

• Figure 1: Likelihood level adapted estimation of marginal likelihood. Note that, $\lambda_1<\lambda_2<\dots<\lambda_{n-1}<\lambda_n$.
• Figure 2: Likelihood level adapted importance sampling (LLA-IS) method: the iso-likelihood contours are shown with $\lambda_1<\lambda_2<\dots<\lambda_n$; importance densities are formed successively to generate samples from high likelihood region.
• Figure 3: Likelihood level adapted stratified sampling (LLA-SS) method: the iso-likelihood contours are shown with $\lambda_1<\lambda_2<\dots<\lambda_n$; more samples are generated from the strata with high likelihood values. For example, the shaded strata with the thick boundary lines have at least one sample with ${\mathcal{L}}(\boldsymbol{\theta})>\lambda_i$.
• Figure 4: Likelihood level adapted particle approximation (LLA-MCMC) method: the iso-likelihood contours are shown with $\lambda_1<\lambda_2<\dots<\lambda_n$; Markov chains are run to generate samples with ${\mathcal{L}}(\boldsymbol{\theta})>\lambda_i$ starting from a sample with ${\mathcal{L}}(\boldsymbol{\theta})>\lambda_{i-1}$.
• Figure 5: The error in the estimation of log evidence is reduced with increasing numbers of likelihood function evaluations for the three proposed algorithms. Note that the three vertical scales are not the same.