The Fundamental Subspaces of Ensemble Kalman Inversion

TL;DR

This work analyzes Ensemble Kalman Inversion (EKI) for linear forward operators by introducing a spectral framework of six fundamental subspaces across observation and state spaces. Through fully discrete analyses of both deterministic and stochastic EKIs, it derives invariant subspaces and convergence rates tied to the standard minimum-norm weighted least-squares solution, showing that only the observable-populated subspace experiences decay at roughly $1/\sqrt{i}$ while other subspaces remain at their initial values. The stochastic analysis uses an idealized covariance iteration to reveal conditions under which small ensembles fail to converge, linking covariance inflation intuition to rigorous subspace behavior. Numerical experiments corroborate the theoretical predictions, illustrating subspace-specific convergence and the critical role of ensemble size. The results offer a principled, subspace-centric view of EKI convergence with potential implications for algorithm design and acceleration in large-scale inverse problems.

Abstract

Ensemble Kalman Inversion (EKI) methods are a family of iterative methods for solving weighted least-squares problems, especially those arising in scientific and engineering inverse problems in which unknown parameters or states are estimated from observed data by minimizing the weighted square norm of the data misfit. Implementation of EKI requires only evaluation of the forward model mapping the unknown to the data, and does not require derivatives or adjoints of the forward model. The methods therefore offer an attractive alternative to gradient-based optimization approaches in inverse problem settings where evaluating derivatives or adjoints of the forward model is computationally intractable. This work presents a new analysis of the behavior of both deterministic and stochastic versions of basic EKI for linear observation operators, resulting in a natural interpretation of EKI's convergence properties in terms of ``fundamental subspaces'' analogous to Strang's fundamental subspaces of linear algebra. Our analysis directly examines the discrete EKI iterations instead of their continuous-time limits considered in previous analyses, and provides spectral decompositions that define six fundamental subspaces of EKI spanning both observation and state spaces. This approach verifies convergence rates previously derived for continuous-time limits, and yields new results describing both deterministic and stochastic EKI convergence behavior with respect to the standard minimum-norm weighted least squares solution in terms of the fundamental subspaces. Numerical experiments illustrate our theoretical results.

Figures (3)

• Figure 1: The four fundamental subspaces of the weighted least squares problem \ref{['eq: least squares problem']} for linear observers, $\mathbf{H}\in\mathbb{R}^{n\times d}$. In observation space (left), the image of the least squares solution (orange star) is the $\boldsymbol{\Gamma}^{-1}$-orthogonal projection of the data (gray star) onto $\textsf{Ran}(\mathbf{H})$. In state space (right), the minimum-norm least squares solution (orange star) lies in $\textsf{Ran}(\mathbf{H}^\top)$ with zero component in $\textsf{Ker}(\mathbf{H})$.
• Figure 2: The six fundamental subspaces of Ensemble Kalman Inversion. In observation/state space (left/right), the three fundamental subspaces are (i) $\textsf{Ran}(\boldsymbol{\mathcal{P}})$/$\textsf{Ran}(\mathbf{P})$, associated with observable populated directions, (ii) $\textsf{Ran}(\boldsymbol{\mathcal{Q}})$/$\textsf{Ran}(\mathbf{Q})$ associated with observable unpopulated directions, and (iii) $\textsf{Ran}(\boldsymbol{\mathcal{N}})$/$\textsf{Ran}(\mathbf{N})$ associated with unobservable directions. In observation space, the image of the least squares solution (orange star) is projected by the oblique projector $\boldsymbol{\mathcal{P}}$ to obtain the EKI solution (blue star). In state space, projection of the minimum norm solution (orange star) by the oblique projector $\mathbf{P}$ yields one possible EKI solution (large blue star) out of the EKI solution manifold (dashed blue line with small blue stars).
• Figure 3: Evolution of particle misfit/residual components under deterministic and stochastic variants of EKI for a randomly generated problem with dimensions $n = 500$ and $d = 1000$. The large ensemble uses $J = 5000$ particles; the small ensemble uses $J = 10$.

Theorems & Definitions (55)

• Proposition 3.1
• proof
• Proposition 3.2
• proof
• Remark 3.3
• Proposition 3.4
• proof
• Remark 3.5
• Lemma 3.6
• proof