Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Early Stopping for Ensemble Kalman-Bucy Inversion

Maia Tienstra, Sebastian Reich

TL;DR

The paper addresses adaptive prior scale selection in Bayesian inverse problems by applying a discrepancy-based early stopping rule. By reparameterizing the problem and formulating it within a time-continuous Ensemble Kalman-Bucy framework, it establishes minimax rates and posterior contraction for the transformed parameter, and demonstrates faithful frequentist coverage of stopped posteriors. It also provides a practical EnKBF-based Monte Carlo algorithm that extends to nonlinear forward maps, with numerical demonstrations showing adaptive performance to signal smoothness and robustness under noise. The results clarify how stopping rules influence estimation accuracy and uncertainty quantification, offering a principled method for adaptive regularization in high-dimensional Bayesian inverse problems.

Abstract

Bayesian linear inverse problems aim to recover an unknown signal from noisy observations, incorporating prior knowledge. This paper analyses a data-dependent method to choose the scale parameter of a Gaussian prior. The method we study arises from early stopping methods, which have been successfully applied to a range of problems, such as statistical inverse problems, in the frequentist setting. These results are extended to the Bayesian setting. We study the use of a discrepancy-based stopping rule in the setting of random noise, which allows for adaptation. Our proposed stopping rule results in optimal rates for the reparameterized problem under certain conditions on the prior covariance operator. We furthermore derive for which class of signals this method is adaptive. It is also shown that the associated posterior contracts at the same rate as the MAP estimator and provides a conservative measure of uncertainty. We implement the proposed stopping rule using the continuous-time ensemble Kalman--Bucy filter (EnKBF). The fictitious time parameter replaces the scale parameter, and the ensemble size is appropriately adjusted in order not to lose the statistical optimality of the computed estimator. With this Monte Carlo algorithm, we extend our results numerically to a nonlinear problem.

Early Stopping for Ensemble Kalman-Bucy Inversion

TL;DR

The paper addresses adaptive prior scale selection in Bayesian inverse problems by applying a discrepancy-based early stopping rule. By reparameterizing the problem and formulating it within a time-continuous Ensemble Kalman-Bucy framework, it establishes minimax rates and posterior contraction for the transformed parameter, and demonstrates faithful frequentist coverage of stopped posteriors. It also provides a practical EnKBF-based Monte Carlo algorithm that extends to nonlinear forward maps, with numerical demonstrations showing adaptive performance to signal smoothness and robustness under noise. The results clarify how stopping rules influence estimation accuracy and uncertainty quantification, offering a principled method for adaptive regularization in high-dimensional Bayesian inverse problems.

Abstract

Bayesian linear inverse problems aim to recover an unknown signal from noisy observations, incorporating prior knowledge. This paper analyses a data-dependent method to choose the scale parameter of a Gaussian prior. The method we study arises from early stopping methods, which have been successfully applied to a range of problems, such as statistical inverse problems, in the frequentist setting. These results are extended to the Bayesian setting. We study the use of a discrepancy-based stopping rule in the setting of random noise, which allows for adaptation. Our proposed stopping rule results in optimal rates for the reparameterized problem under certain conditions on the prior covariance operator. We furthermore derive for which class of signals this method is adaptive. It is also shown that the associated posterior contracts at the same rate as the MAP estimator and provides a conservative measure of uncertainty. We implement the proposed stopping rule using the continuous-time ensemble Kalman--Bucy filter (EnKBF). The fictitious time parameter replaces the scale parameter, and the ensemble size is appropriately adjusted in order not to lose the statistical optimality of the computed estimator. With this Monte Carlo algorithm, we extend our results numerically to a nonlinear problem.
Paper Structure (18 sections, 20 theorems, 164 equations, 3 figures, 1 algorithm)

This paper contains 18 sections, 20 theorems, 164 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Previous work on early stopping and ensemble Kalman inversion
  4. Main contributions and paper outline
  5. Further notation
  6. Theoretical results on adaptive estimation
  7. Structural Assumptions
  8. Theoretical Results
  9. Ensemble Kalman--Bucy inversion
  10. Algorithmic details
  11. Numerical Examples
  12. Linear Examples
  13. Non-Linear Example
  14. One dimensional Schrödinger Equation
  15. Numerical Implementation
  16. ...and 3 more sections

Key Result

Proposition 1.1

(Prop. 3.1 in knapik2016) For given $\tau_{n}>0$, the prior distribution for $\theta$ is $\mathcal{N}(0, \tau_{n}^2 C_0)$ and $Y$ given $\theta$ is $\mathcal{N}(G \theta, n^{-1} I)$ distributed. Then the conditional distribution of $\theta$ given $Y$, the posterior, is Gaussian $\mathcal{N}(\widehat and covariance operator where the Kalman gain $K_{\tau_{n}}: H_2 \rightarrow H_1$ is the linear co

Figures (3)

  • Figure 1: Here we plot the results of running \ref{['algo:deter_enkf']} for two different smoothness levels. We stop when $R_n^2 \leq \kappa_{\rm dp} = C D(n) /n$, with $C =1$ for all experiments. The top row has ground truth coefficients $\theta_{{\rm rough},i} = 5 \sin (0.5i) i^{-1}$ and the bottom row as ground truth coefficients $\theta_{{\rm smooth},i} = 5 \text{exp}(-i)$ for $i=1,...,100$. From left to right, we decrease the noise level of the observations, thus simulating an increase in sample size. This is valid as in the linear case, the $ith$ observations are i.i.d. The prior is the same for each experiment with $\alpha =1$, and the forward operator is fixed with $p=1/2$. We see that we can adapt to both functions and that, as the noise level decreases, there is a slight contraction of the ensemble members. The $95\%$ credible region is the shaded region and is estimated by the $95\%$ quantiles of the ensemble members.
  • Figure 2: Here we plot the result of running \ref{['algo:deter_enkf']} on the direct nonlinear problem with the prior specified in \ref{['eq:nonlinear_prior']}. From left to right, we see the noise level decrease and the ensemble particles collapse. The shaded region is the $95\%$ credible region as estimated by the $95\%$ quantiles of the ensemble members. We also see that the mean goes to the ground truth.
  • Figure 3: In the top row, we have plotted the results of running \ref{['algo:deter_enkf']}, with stopping criterion $R \leq C D(n)/n$ where $C=0.1$ when the noise level is $1e-2$, and $C=0.5$ else. The middle row is the resulting HMC estimates. The bottom row is a comparison of the particles as samples from the posterior and the samples from the HMC overlayed. We see that the EnKF overestimates the variance of the posterior, but still maintains the correct shape. For all noise levels, the r-hat value was 1.

Theorems & Definitions (53)

  • Remark 1.1
  • Proposition 1.1
  • Remark 1.2
  • Definition 1.1
  • Definition 1.2
  • Remark 2.1
  • Remark 2.2
  • Remark 2.3
  • Remark 2.4
  • Remark 2.5
  • ...and 43 more