Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learned harmonic mean estimation of the marginal likelihood for multimodal posteriors with flow matching

Alicja Polanska, Jason D. McEwen

TL;DR

The paper tackles the challenge of computing the Bayesian evidence $z$ for complex, multimodal posteriors by integrating flow matching-based continuous normalizing flows (CNFs) into the learned harmonic mean estimator. This yields a flexible, simulation-free target density $\varphi(\theta)$ that concentrates within the posterior, producing finite-variance estimates from posterior samples. The authors demonstrate robustness and accuracy on challenging benchmarks, including the Rastrigin function and a 20‑dimensional Gaussian mixture, without hand-tuned base distributions. The approach promises improved, sampling-agnostic model comparison in high-dimensional Bayesian inference and broad applications in physics and cosmology.

Abstract

The marginal likelihood, or Bayesian evidence, is a crucial quantity for Bayesian model comparison but its computation can be challenging for complex models, even in parameters space of moderate dimension. The learned harmonic mean estimator has been shown to provide accurate and robust estimates of the marginal likelihood simply using posterior samples. It is agnostic to the sampling strategy, meaning that the samples can be obtained using any method. This enables marginal likelihood calculation and model comparison with whatever sampling is most suitable for the task. However, the internal density estimators considered previously for the learned harmonic mean can struggle with highly multimodal posteriors. In this work we introduce flow matching-based continuous normalizing flows as a powerful architecture for the internal density estimation of the learned harmonic mean. We demonstrate the ability to handle challenging multimodal posteriors, including an example in 20 parameter dimensions, showcasing the method's ability to handle complex posteriors without the need for fine-tuning or heuristic modifications to the base distribution.

Learned harmonic mean estimation of the marginal likelihood for multimodal posteriors with flow matching

TL;DR

The paper tackles the challenge of computing the Bayesian evidence for complex, multimodal posteriors by integrating flow matching-based continuous normalizing flows (CNFs) into the learned harmonic mean estimator. This yields a flexible, simulation-free target density that concentrates within the posterior, producing finite-variance estimates from posterior samples. The authors demonstrate robustness and accuracy on challenging benchmarks, including the Rastrigin function and a 20‑dimensional Gaussian mixture, without hand-tuned base distributions. The approach promises improved, sampling-agnostic model comparison in high-dimensional Bayesian inference and broad applications in physics and cosmology.

Abstract

The marginal likelihood, or Bayesian evidence, is a crucial quantity for Bayesian model comparison but its computation can be challenging for complex models, even in parameters space of moderate dimension. The learned harmonic mean estimator has been shown to provide accurate and robust estimates of the marginal likelihood simply using posterior samples. It is agnostic to the sampling strategy, meaning that the samples can be obtained using any method. This enables marginal likelihood calculation and model comparison with whatever sampling is most suitable for the task. However, the internal density estimators considered previously for the learned harmonic mean can struggle with highly multimodal posteriors. In this work we introduce flow matching-based continuous normalizing flows as a powerful architecture for the internal density estimation of the learned harmonic mean. We demonstrate the ability to handle challenging multimodal posteriors, including an example in 20 parameter dimensions, showcasing the method's ability to handle complex posteriors without the need for fine-tuning or heuristic modifications to the base distribution.
Paper Structure (9 sections, 10 equations, 4 figures)

This paper contains 9 sections, 10 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Methodology
  3. Learned harmonic mean estimator
  4. Flow matching
  5. Learned harmonic mean with flow matching
  6. Numerical experiments
  7. Rastrigin example
  8. Gaussian mixture model
  9. Conclusion

Figures (4)

  • Figure 1: Corner plot of samples from the posterior (red) and trained flow at temperature $T=0.98$ (blue) for the Rastrigin example. The target distribution given by the concentrated flow is contained within the posterior as required for the learned harmonic mean estimator. All represented modes are captured by the flow accurately.
  • Figure 2: Marginal likelihood computed by the learned harmonic mean estimator for the Rastrigin example. Violin plot of $100$ runs of the experiment, showing the distribution of marginal likelihood values (measured) along with the error estimate computed by harmonic (estimated). The ground truth obtained directly by numerical integration is indicated by the red dashed line.
  • Figure 3: Corner plot for the first five dimensions of samples from the posterior (red) and trained flow with temperature $T=0.95$ (blue). The target distribution given by the concentrated flow is concentrated within the posterior, as required for the learned harmonic mean estimator. The challenging topology is correctly captured by the flow.
  • Figure 4: Marginal likelihood computed by the learned harmonic mean estimator for the mixture of five Gaussians in $20$ dimensions. Violin plot of $100$ runs of the experiment, showing the distribution of marginal likelihood values (measured) along with the error estimate computed by harmonic (estimated). The analytic ground truth is indicated by the red dashed line.