Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ensemble Kalman Inversion: mean-field limit and convergence analysis

Zhiyan Ding, Qin Li

TL;DR

This work analyzes Ensemble Kalman Inversion (EKI) as a sampling method for Bayesian inverse problems by studying the continuous-time, J-particle limit. It proves a mean-field limit: as J → ∞, the empirical measure of the coupled SDEs governing the particles converges to the solution of a Fokker-Planck equation in the Wasserstein-2 metric, with a rate of order J^{−1/2+ε} in low dimensions and dimension-aware behavior for higher dimensions. In the linear setting (G(u) = Au), the FP solution coincides with the posterior, yielding finite-time reconstruction of the target distribution; in the nonlinear, weakly nonlinear setting, the FP limit remains well-posed and captures the dominant dynamics, with additional weight terms accounting for nonlinearity. The analysis employs a bridge SDE to connect particle dynamics with the FP flow, leverages Bennett–Davies–Fournier-type moment bounds, and uses bootstrapping to obtain optimal convergence rates, providing rigorous justification for EKI as a high-dimensional sampling tool with explicit convergence guarantees.

Abstract

Ensemble Kalman Inversion (EKI) has been a very popular algorithm used in Bayesian inverse problems. It samples particles from a prior distribution, and introduces a motion to move the particles around in pseudo-time. As the pseudo-time goes to infinity, the method finds the minimizer of the objective function, and when the pseudo-time stops at $1$, the ensemble distribution of the particles resembles, in some sense, the posterior distribution in the linear setting. The ideas trace back further to Ensemble Kalman Filter and the associated analysis, but to today, when viewed as a sampling method, why EKI works, and in what sense with what rate the method converges is still largely unknown. In this paper, we analyze the continuous version of EKI, a coupled SDE system, and prove the mean field limit of this SDE system. In particular, we will show that 1. as the number of particles goes to infinity, the empirical measure of particles following SDE converges to the solution to a Fokker-Planck equation in Wasserstein 2-distance with an optimal rate, for both linear and weakly nonlinear case; 2. the solution to the Fokker-Planck equation reconstructs the target distribution in finite time in the linear case.

Ensemble Kalman Inversion: mean-field limit and convergence analysis

TL;DR

This work analyzes Ensemble Kalman Inversion (EKI) as a sampling method for Bayesian inverse problems by studying the continuous-time, J-particle limit. It proves a mean-field limit: as J → ∞, the empirical measure of the coupled SDEs governing the particles converges to the solution of a Fokker-Planck equation in the Wasserstein-2 metric, with a rate of order J^{−1/2+ε} in low dimensions and dimension-aware behavior for higher dimensions. In the linear setting (G(u) = Au), the FP solution coincides with the posterior, yielding finite-time reconstruction of the target distribution; in the nonlinear, weakly nonlinear setting, the FP limit remains well-posed and captures the dominant dynamics, with additional weight terms accounting for nonlinearity. The analysis employs a bridge SDE to connect particle dynamics with the FP flow, leverages Bennett–Davies–Fournier-type moment bounds, and uses bootstrapping to obtain optimal convergence rates, providing rigorous justification for EKI as a high-dimensional sampling tool with explicit convergence guarantees.

Abstract

Ensemble Kalman Inversion (EKI) has been a very popular algorithm used in Bayesian inverse problems. It samples particles from a prior distribution, and introduces a motion to move the particles around in pseudo-time. As the pseudo-time goes to infinity, the method finds the minimizer of the objective function, and when the pseudo-time stops at , the ensemble distribution of the particles resembles, in some sense, the posterior distribution in the linear setting. The ideas trace back further to Ensemble Kalman Filter and the associated analysis, but to today, when viewed as a sampling method, why EKI works, and in what sense with what rate the method converges is still largely unknown. In this paper, we analyze the continuous version of EKI, a coupled SDE system, and prove the mean field limit of this SDE system. In particular, we will show that 1. as the number of particles goes to infinity, the empirical measure of particles following SDE converges to the solution to a Fokker-Planck equation in Wasserstein 2-distance with an optimal rate, for both linear and weakly nonlinear case; 2. the solution to the Fokker-Planck equation reconstructs the target distribution in finite time in the linear case.

Paper Structure

This paper contains 17 sections, 16 theorems, 167 equations, 1 algorithm.

Table of Contents

  1. Introduction
  2. Ensemble Kalman Inversion setup and statement of our result
  3. Ensemble Kalman Inversion
  4. Continuum limit and dynamical system of $\{u_t^j\}$
  5. Main theorem and mean field limit
  6. Linear and nonlinear setups
  7. Linear setup
  8. Nonlinear setup
  9. Proof of Theorem \ref{['thm:mean_field']}, Part I
  10. Properties of the Fokker-Planck equation
  11. $\{v^j\}$ and the Fokker-Planck-like equation
  12. Proof of Theorem \ref{['thm:mean_field']} Part II, and Theorem \ref{['thm:moments']}
  13. Some a-priori estimates
  14. Proof of Proposition \ref{['thm:vj_uj']}
  15. Acknowledgement
  16. ...and 2 more sections

Key Result

theorem 1

Under the weakly nonlinear assumption linear, the mean field limit of $M_{u_t}$ is the probability distribution induced by $\rho(t,u)$. Here $M_{u_t}$ is the ensemble distribution of $\{u^j_t\}$ as defined in eqn:ensemble and $\rho(t,u)$ is the strong solution to the following Fokker-Planck equation where $\mu_0$ is the prior density function, $\mathcal{H}_u(\rho)$ is Hessian of $\rho$. More speci

Theorems & Definitions (30)

  • remark thmcounterremark
  • definition thmcounterdefinition
  • theorem 1: Main result 1: mean-field limit
  • theorem 2: Main result 2: weak convergence
  • corollary thmcountercorollary
  • lemma thmcounterlemma
  • proof
  • lemma thmcounterlemma
  • proof
  • proposition thmcounterproposition: Linking $\{v^j\}$ with Fokker-Planck-like PDE
  • ...and 20 more