Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ensemble Kalman inversion with non-smooth regularization

Simon Weissmann

Abstract

This paper investigates ensemble Kalman inversion (EKI) for variational inverse problems with convex, potentially non-smooth regularization. While deterministic EKI and its Tikhonov-regularized variants have primarily been analyzed for smooth objectives, a corresponding framework accommodating subgradient dynamics has not yet been established. To address this gap, we introduce a subgradient-based formulation of EKI (SEKI) that incorporates non-smooth regularizers through a covariance-preconditioned differential inclusion for the ensemble mean. In the linear forward-model setting, well-posedness of the resulting continuous-time particle system is established under minimal assumptions on the regularization functional using maximal monotone operator theory and Yosida approximations. Motivated by the continuous-time dynamics, we propose an explicit discrete-time scheme that preserves the derivative-free structure of EKI and analyze its convergence as an optimization method in the strongly convex case. Numerical experiments in computed tomography with total variation regularization and sparse recovery with $\ell_1$ penalties illustrate that non-smooth regularization can be incorporated into ensemble Kalman inversion in a stable and principled manner.

Ensemble Kalman inversion with non-smooth regularization

Abstract

This paper investigates ensemble Kalman inversion (EKI) for variational inverse problems with convex, potentially non-smooth regularization. While deterministic EKI and its Tikhonov-regularized variants have primarily been analyzed for smooth objectives, a corresponding framework accommodating subgradient dynamics has not yet been established. To address this gap, we introduce a subgradient-based formulation of EKI (SEKI) that incorporates non-smooth regularizers through a covariance-preconditioned differential inclusion for the ensemble mean. In the linear forward-model setting, well-posedness of the resulting continuous-time particle system is established under minimal assumptions on the regularization functional using maximal monotone operator theory and Yosida approximations. Motivated by the continuous-time dynamics, we propose an explicit discrete-time scheme that preserves the derivative-free structure of EKI and analyze its convergence as an optimization method in the strongly convex case. Numerical experiments in computed tomography with total variation regularization and sparse recovery with penalties illustrate that non-smooth regularization can be incorporated into ensemble Kalman inversion in a stable and principled manner.
Paper Structure (30 sections, 8 theorems, 126 equations, 9 figures, 2 tables, 1 algorithm)

This paper contains 30 sections, 8 theorems, 126 equations, 9 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Contributions
  4. Outline.
  5. Mathematical setup
  6. Inverse problem
  7. Ensemble Kalman inversion
  8. Subgradient ensemble Kalman inversion
  9. Linear setting
  10. Well-posedness of the SEKI flow in the linear setting
  11. Sample covariance bounds
  12. Uniqueness of solutions
  13. Yosida approximation
  14. Proof of \ref{['thm:main_existence']}
  15. From continous- to discrete-time: Subgradient ensemble Kalman inversion
  16. ...and 15 more sections

Key Result

Theorem 3.1

Let $G(\cdot)= A\cdot$ for some matrix $A\in\mathbb{R}^{K\times d}$ and suppose that ass:regularization is in place. Let $x^{(j)}(0) = x_0^{(j)}\in\mathbb{R}^d$, $j=1,\dots,J$ be such that the initial covariance $C(0)=C(x_0)$ is positive definite. Then for every $T>0$ the system eq:SEKI_flow_linear

Figures (9)

  • Figure 1: Reconstruction of the unknown image $x^\dagger$ using Sub-GD and SEKI-f ($k_b=5000$) after $5\cdot10^5$ iterations. From left to right: ground truth, Sub-GD reconstruction, SEKI-f reconstruction (ensemble mean), and the reference solution $x_\ast$.
  • Figure 2: Objective gap $\Phi_R(x_k)-\Phi_R(x_\ast)$ for Sub-GD and SEKI-f with burn-in lengths $k_b\in\{50,1000,2000,5000\}$. Left: convergence versus iteration. Right: convergence versus computational time.
  • Figure 3: Relative reconstruction error $\|\bar{x}_k-x_\ast\|/\|x_\ast\|$ for Sub-GD and SEKI-f with burn-in lengths $k_b\in\{50,1000,2000,5000\}$. Left: error versus iteration. Right: error versus computational time.
  • Figure 4: Relative reconstruction error (left) and objective gap (right) for the compressed sensing experiment with correlation parameter $\rho=0$. Results are shown as a function of computational time for Sub-GD and SEKI-f with burn-in lengths $k_b\in\{500,2000,4000\}$.
  • Figure 5: Relative reconstruction error (left) and objective gap (right) for the compressed sensing experiment with correlation parameter $\rho=0.95$. Results are shown as a function of computational time for Sub-GD and SEKI-f with burn-in lengths $k_b\in\{500,2000,4000\}$.
  • ...and 4 more figures

Theorems & Definitions (19)

  • Theorem 3.1: Well-posedness
  • Remark 3.2
  • Lemma 3.3: Sample covariance representation
  • proof
  • Proposition 3.4: Uniqueness of solutions
  • proof
  • Remark 3.5
  • Lemma 3.6: Uniform boundedness
  • proof
  • Lemma 3.7: Cauchy sequence
  • ...and 9 more