Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Investigation into the Distribution of Ratios of Particle Solver-based Likelihoods

Emil Løvbak, Sebastian Krumscheid

TL;DR

The paper addresses Bayesian inference for a diffusion-model forward map that is stochastic due to Monte Carlo solvers. It analyzes how randomness in the forward map propagates into Metropolis-Hastings acceptance by studying the distribution of ratios of approximate likelihoods and derives a moment-existence condition: the $p$-th moment exists iff $p\,\Sigma_\eta \succ \Sigma(D_2)$. An explicit moment formula is provided and validated through numerical experiments on a 1D diffusion problem, showing that when forward-model variance is large the acceptance behavior can degrade to an uninformative 0.5 bound, while for smaller variance the MH steps remain informative. The results offer practical guidance on selecting the Monte Carlo particle count $P$ and understanding acceptance-rate behavior under solver noise, with potential to bound acceptance errors using the derived moments.

Abstract

We investigate the use of the Metropolis-Hastings algorithm to sample posterior distribution in a Bayesian inverse problem, where the likelihood function is random. Concretely, we consider the case where one has full field observations of a PDE solution, in case a one-dimensional diffusion equation, subject to a Gaussian observation error. Assuming one uses a particle-based Monte Carlo simulation when approximating the likelihood function, one gets an approximate likelihood with additive Gaussian noise in the log-likelihood. We study how these two Gaussian distributions affect the distribution of ratios of approximate likelihood evaluations, as required when evaluating acceptance probabilities in the Metropolis-Hastings algorithm. We do so through both theoretical analysis and numerical experiments.

An Investigation into the Distribution of Ratios of Particle Solver-based Likelihoods

TL;DR

The paper addresses Bayesian inference for a diffusion-model forward map that is stochastic due to Monte Carlo solvers. It analyzes how randomness in the forward map propagates into Metropolis-Hastings acceptance by studying the distribution of ratios of approximate likelihoods and derives a moment-existence condition: the -th moment exists iff . An explicit moment formula is provided and validated through numerical experiments on a 1D diffusion problem, showing that when forward-model variance is large the acceptance behavior can degrade to an uninformative 0.5 bound, while for smaller variance the MH steps remain informative. The results offer practical guidance on selecting the Monte Carlo particle count and understanding acceptance-rate behavior under solver noise, with potential to bound acceptance errors using the derived moments.

Abstract

We investigate the use of the Metropolis-Hastings algorithm to sample posterior distribution in a Bayesian inverse problem, where the likelihood function is random. Concretely, we consider the case where one has full field observations of a PDE solution, in case a one-dimensional diffusion equation, subject to a Gaussian observation error. Assuming one uses a particle-based Monte Carlo simulation when approximating the likelihood function, one gets an approximate likelihood with additive Gaussian noise in the log-likelihood. We study how these two Gaussian distributions affect the distribution of ratios of approximate likelihood evaluations, as required when evaluating acceptance probabilities in the Metropolis-Hastings algorithm. We do so through both theoretical analysis and numerical experiments.

Paper Structure

This paper contains 7 sections, 1 theorem, 26 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Monte Carlo simulation of the diffusion equation
  3. Bias.
  4. Variance.
  5. Analysis of approximate likelihood ratios
  6. Numerical experiments
  7. Conclusions

Key Result

theorem thmcountertheorem

Given two parameter values $D_1, D_2 \in \mathbb{R}$ and an approximate forward map $\hat{G}: \mathbb{R} \mapsto \mathbb{R}^N$ such that $\hat{G}[D] = G[D] + \delta$ with $\delta \overset{\text{i.i.d.}}{\sim} \mathcal{N}(\mu(D), \Sigma(D))$ where $\mu: \mathbb{R} \mapsto \mathbb{R}^N$ and $\Sigma = is a random variable, whose $p$-th moment exists iff $p\Sigma_\eta \succ \Sigma(D_2)$, with $\succ$

Figures (4)

  • Figure 1: Computing the expectation of the approximate likelihoods and the ratio of these expectations, given a discrete solution of the problem \ref{['eq:diff_eq']}--\ref{['eq:diff_eq_boundary']} perturbed by synthetic observation noise, for $D^\ast = D_1 =0.1$ and $D_2 = 0.08$.
  • Figure 2: Computing the expectation of the approximate likelihood ratio given a discrete solution of the problem \ref{['eq:diff_eq']}--\ref{['eq:diff_eq_boundary']} perturbed by synthetic observation noise for $D^\ast = D_1 = D_2 =0.1$.
  • Figure 3: Computing the expectation of the approximate likelihood ratio given a discrete solution of the problem \ref{['eq:diff_eq']}--\ref{['eq:diff_eq_boundary']} perturbed by synthetic observation noise for $D^\ast = D_2 =0.1$ and $D_1 = 0.08$.
  • Figure 4: Computing the expectation of the approximate likelihood ratio given a discrete solution of the problem \ref{['eq:diff_eq']}--\ref{['eq:diff_eq_boundary']} perturbed by synthetic observation noise for $D^\ast = D_1 =0.1$ and $D_2=0.08$.

Theorems & Definitions (4)

  • theorem thmcountertheorem
  • proof
  • remark thmcounterremark
  • remark thmcounterremark