Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mixed Noise and Posterior Estimation with Conditional DeepGEM

Paul Hagemann, Johannes Hertrich, Maren Casfor, Sebastian Heidenreich, Gabriele Steidl

TL;DR

The paper tackles Bayesian inverse problems under a mixed Gaussian noise model for indirect measurements by proposing a nested EM framework that couples a conditional normalizing flow-based E-step with an analytic inner EM for noise parameters $(a,b)$. By training with the forward KL, the approach achieves mode-covering posteriors and efficiently leverages information from multiple measurements, outperforming reverse KL-based variants in multimodal settings. The method is instantiated as a conditional DeepGEM and validated on EUV scatterometry tasks in nanometrology, demonstrating improved posterior reconstructions and reliable noise parameter estimation. This work enables fast, multi-measurement Bayesian inference under complex noise models, with potential for real-time applications in nanometrology and related indirect-measurement domains.

Abstract

Motivated by indirect measurements and applications from nanometrology with a mixed noise model, we develop a novel algorithm for jointly estimating the posterior and the noise parameters in Bayesian inverse problems. We propose to solve the problem by an expectation maximization (EM) algorithm. Based on the current noise parameters, we learn in the E-step a conditional normalizing flow that approximates the posterior. In the M-step, we propose to find the noise parameter updates again by an EM algorithm, which has analytical formulas. We compare the training of the conditional normalizing flow with the forward and reverse KL, and show that our model is able to incorporate information from many measurements, unlike previous approaches.

Mixed Noise and Posterior Estimation with Conditional DeepGEM

TL;DR

The paper tackles Bayesian inverse problems under a mixed Gaussian noise model for indirect measurements by proposing a nested EM framework that couples a conditional normalizing flow-based E-step with an analytic inner EM for noise parameters . By training with the forward KL, the approach achieves mode-covering posteriors and efficiently leverages information from multiple measurements, outperforming reverse KL-based variants in multimodal settings. The method is instantiated as a conditional DeepGEM and validated on EUV scatterometry tasks in nanometrology, demonstrating improved posterior reconstructions and reliable noise parameter estimation. This work enables fast, multi-measurement Bayesian inference under complex noise models, with potential for real-time applications in nanometrology and related indirect-measurement domains.

Abstract

Motivated by indirect measurements and applications from nanometrology with a mixed noise model, we develop a novel algorithm for jointly estimating the posterior and the noise parameters in Bayesian inverse problems. We propose to solve the problem by an expectation maximization (EM) algorithm. Based on the current noise parameters, we learn in the E-step a conditional normalizing flow that approximates the posterior. In the M-step, we propose to find the noise parameter updates again by an EM algorithm, which has analytical formulas. We compare the training of the conditional normalizing flow with the forward and reverse KL, and show that our model is able to incorporate information from many measurements, unlike previous approaches.
Paper Structure (17 sections, 2 theorems, 43 equations, 6 figures, 4 tables, 1 algorithm)

This paper contains 17 sections, 2 theorems, 43 equations, 6 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions
  3. Organization
  4. EM Algorithm
  5. Parameter Estimation in Bayesian Inverse Problems
  6. E-Step: Conditional NFs
  7. M-Step: Inner EM for Mixed Noise Model
  8. Inner E-step
  9. Inner M-step:
  10. Resulting Algorithm
  11. Experiments
  12. Models evaluated
  13. Metrics
  14. Conclusions and Limitations
  15. Derivation of the inner M-step
  16. ...and 2 more sections

Key Result

Lemma 1

Let $X \in \mathbb{R}^d$ be a absolute continuous random variable and let $Q$ be an absolutely continuous measure on $\mathbb{R}^d$ with probability density function $q$. Then it holds, for any $y\in\mathbb{R}^n$, that

Figures (6)

  • Figure 1: Distance to the hyperparameters (a,b) for forward and reverse KL conditional DeepGEM.
  • Figure 2: Posterior reconstructions for different measurements using forward/reverse conditional DeepGEM via one dimensional histograms on the diagonal and two dimensional on the offdiagonal. Ground truth x is depicted by the blue line.
  • Figure 3: Distance to the hyperparameters (a,b) for forward and reverse KL conditional DeepGEM.
  • Figure 4: Posterior reconstructions for different measurements using forward/reverse conditional DeepGEM.
  • Figure 5: Convergence plots for $a$ and $b$ where we save every 20 EM-steps.
  • ...and 1 more figures

Theorems & Definitions (5)

  • Lemma 1
  • proof
  • Proposition 2
  • Remark 3: Generalized EM algorithms
  • Remark 4: Forward versus Reverse KL