Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximate Bayesian Computation with Statistical Distances for Model Selection

Clara Grazian

TL;DR

Overall, the study clarifies the strengths and limitations of discrepancy-based ABC for likelihood-free model choice and provides guidance for its use in realistic inferential settings.

Abstract

Model selection in the presence of intractable likelihoods remains a central challenge in Bayesian inference. Approximate Bayesian computation (ABC) provides a flexible likelihood-free framework, but its use for model choice is known to be sensitive to the choice of summary statistics, often leading to poorly calibrated posterior model probabilities. Recent ABC variants based on statistical distances allow comparisons to be performed directly on empirical distributions, avoiding data reduction and offering improved theoretical guarantees under suitable conditions. This paper provides a systematic evaluation of discrepancy-based ABC methods for Bayesian model selection, focusing on their empirical behavior across a range of simulation settings and levels of model complexity. We compare full data ABC approaches based on Wasserstein, Creamer-von-Mises, and maximum mean discrepancy metrics with summary-statistic-based ABC and neural network classifiers. The results highlight settings in which full data ABC yields stable and well-calibrated posterior model probabilities, as well as scenarios where performance degrades due to model overlap or dependence. An application to toad movement models illustrates the practical implications of these findings. Overall, the study clarifies the strengths and limitations of discrepancy-based ABC for likelihood-free model choice and provides guidance for its use in realistic inferential settings.

Approximate Bayesian Computation with Statistical Distances for Model Selection

TL;DR

Overall, the study clarifies the strengths and limitations of discrepancy-based ABC for likelihood-free model choice and provides guidance for its use in realistic inferential settings.

Abstract

Model selection in the presence of intractable likelihoods remains a central challenge in Bayesian inference. Approximate Bayesian computation (ABC) provides a flexible likelihood-free framework, but its use for model choice is known to be sensitive to the choice of summary statistics, often leading to poorly calibrated posterior model probabilities. Recent ABC variants based on statistical distances allow comparisons to be performed directly on empirical distributions, avoiding data reduction and offering improved theoretical guarantees under suitable conditions. This paper provides a systematic evaluation of discrepancy-based ABC methods for Bayesian model selection, focusing on their empirical behavior across a range of simulation settings and levels of model complexity. We compare full data ABC approaches based on Wasserstein, Creamer-von-Mises, and maximum mean discrepancy metrics with summary-statistic-based ABC and neural network classifiers. The results highlight settings in which full data ABC yields stable and well-calibrated posterior model probabilities, as well as scenarios where performance degrades due to model overlap or dependence. An application to toad movement models illustrates the practical implications of these findings. Overall, the study clarifies the strengths and limitations of discrepancy-based ABC for likelihood-free model choice and provides guidance for its use in realistic inferential settings.

Paper Structure

This paper contains 17 sections, 2 theorems, 46 equations, 23 figures, 6 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Notation and Setting
  4. Approximate Bayesian Computation
  5. Approximate Bayesian Computation for Model Selection
  6. ABC-MC with Statistical Distances
  7. Theoretical justification
  8. Simulation Study
  9. Normal Mean Hypothesis Test
  10. Exponential Family Model Selection
  11. Computational Performance Comparison
  12. $g$-and-$k$ Distribution Model Selection
  13. Toad Movement Model Selection
  14. Real Data Example
  15. Conclusion
  16. ...and 2 more sections

Key Result

Theorem 3.1

Assume: Then, for any sequence $\varepsilon_n \to 0$ such that $\varepsilon_n < \min_{k^*} \delta_{k^*} / 2$,

Figures (23)

  • Figure 1: Posterior distributions of model $M_3$ parameters across the considered methods.
  • Figure 2: Confusion matrices showing model selection performance for each method in the normal example with $n=100$.
  • Figure 3: Boxplots of the estimated posterior probability of $H_0$ across 100 datasets generated from $N(0,1)$.
  • Figure 4: Boxplots of the estimated posterior probability of $H_0$ across 100 datasets generated from $N(0.1,1)$.
  • Figure 5: Boxplots of the estimated posterior probability of $H_0$ across 100 datasets generated from $N(0.2,1)$.
  • ...and 18 more figures

Theorems & Definitions (4)

  • Theorem 3.1: Model Selection Consistency of ABC-Wass
  • Theorem 3.2: Robustness to Model Misspecification
  • proof
  • proof