Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ensemble Kalman Inversion for nonlinear problems: weights, consistency, and variance bounds

Zhiyan Ding, Qin Li, Jianfeng Lu

TL;DR

This work tackles the bias of Ensemble Kalman Inversion (EKI) and Ensemble Square Root Filter (EnSRF) in nonlinear settings and the high variance of Importance Sampling (IS). It introduces Weighted EnKI (WEnKI) and Weighted EnSRF (WEnSRF), which add a time-evolving weight to enforce a flow from the prior to the posterior, yielding unbiased sampling with bounded weight variance. The authors prove consistency and derive variance-bounding results under two nonlinearity regimes, and connect the weighted flow to a Fokker-Planck framework to explain when EnKI performs well in practice. Numerical tests in 1D and 2D nonlinear inverse problems illustrate that WEnKI/WEnSRF outperform classical methods and IS in capturing non-Gaussian, multimodal posteriors while maintaining controlled weight variance, albeit with higher computational cost due to derivative-enabled weight terms.

Abstract

Ensemble Kalman Inversion (EnKI) and Ensemble Square Root Filter (EnSRF) are popular sampling methods for obtaining a target posterior distribution. They can be seem as one step (the analysis step) in the data assimilation method Ensemble Kalman Filter. Despite their popularity, they are, however, not unbiased when the forward map is nonlinear. Important Sampling (IS), on the other hand, obtains the unbiased sampling at the expense of large variance of weights, leading to slow convergence of high moments. We propose WEnKI and WEnSRF, the weighted versions of EnKI and EnSRF in this paper. It follows the same gradient flow as that of EnKI/EnSRF with weight corrections. Compared to the classical methods, the new methods are unbiased, and compared with IS, the method has bounded weight variance. Both properties will be proved rigorously in this paper. We further discuss the stability of the underlying Fokker-Planck equation. This partially explains why EnKI, despite being inconsistent, performs well occasionally in nonlinear settings. Numerical evidence will be demonstrated at the end.

Ensemble Kalman Inversion for nonlinear problems: weights, consistency, and variance bounds

TL;DR

This work tackles the bias of Ensemble Kalman Inversion (EKI) and Ensemble Square Root Filter (EnSRF) in nonlinear settings and the high variance of Importance Sampling (IS). It introduces Weighted EnKI (WEnKI) and Weighted EnSRF (WEnSRF), which add a time-evolving weight to enforce a flow from the prior to the posterior, yielding unbiased sampling with bounded weight variance. The authors prove consistency and derive variance-bounding results under two nonlinearity regimes, and connect the weighted flow to a Fokker-Planck framework to explain when EnKI performs well in practice. Numerical tests in 1D and 2D nonlinear inverse problems illustrate that WEnKI/WEnSRF outperform classical methods and IS in capturing non-Gaussian, multimodal posteriors while maintaining controlled weight variance, albeit with higher computational cost due to derivative-enabled weight terms.

Abstract

Ensemble Kalman Inversion (EnKI) and Ensemble Square Root Filter (EnSRF) are popular sampling methods for obtaining a target posterior distribution. They can be seem as one step (the analysis step) in the data assimilation method Ensemble Kalman Filter. Despite their popularity, they are, however, not unbiased when the forward map is nonlinear. Important Sampling (IS), on the other hand, obtains the unbiased sampling at the expense of large variance of weights, leading to slow convergence of high moments. We propose WEnKI and WEnSRF, the weighted versions of EnKI and EnSRF in this paper. It follows the same gradient flow as that of EnKI/EnSRF with weight corrections. Compared to the classical methods, the new methods are unbiased, and compared with IS, the method has bounded weight variance. Both properties will be proved rigorously in this paper. We further discuss the stability of the underlying Fokker-Planck equation. This partially explains why EnKI, despite being inconsistent, performs well occasionally in nonlinear settings. Numerical evidence will be demonstrated at the end.

Paper Structure

This paper contains 22 sections, 12 theorems, 162 equations, 2 figures, 3 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Importance Sampling and ensemble Kalman filter
  3. Importance sampling
  4. Ensemble Kalman filters
  5. Ensemble Square Root filter
  6. Ensemble Kalman Inversion
  7. Summary
  8. Weighted Ensemble Kalman Inversion and Square root filter
  9. Weighted ensemble square root filter (WEnSRF)
  10. Weighted Ensemble Kalman Inversion (WEnKI)
  11. Properties of WEnKI and WEnSRF
  12. Consistency
  13. Bounding the variance of weights
  14. Bounded nonlinearity under Assumption \ref{['Gassum1']}
  15. Weak nonlinearity under Assumption \ref{['Gassum2']}
  16. ...and 7 more sections

Key Result

Theorem 4.1

Assume $\mathcal{G}:\mathbb{R}^L\rightarrow \mathbb{R}^K$ is a $C^1$ function, then:

Figures (2)

  • Figure 4: Example 3: $\log(\textrm{Var}(Nw(t))+1)$ for WEnKI, WEnSRF and IS.
  • Figure 7: Example 5: $\log(\textrm{Var}(Nw(t))+1)$ for WEnKI, WEnSRF and IS

Theorems & Definitions (26)

  • Remark 1
  • Remark 2
  • Theorem 4.1
  • proof
  • Theorem 4.2
  • proof
  • Lemma 4.3
  • proof
  • Lemma 4.4
  • proof
  • ...and 16 more