Stochastic Gradient Descent for Nonlinear Inverse Problems in Banach Spaces

TL;DR

The paper develops stochastic gradient descent methods for solving nonlinear inverse problems in Banach spaces within an iterative-regularization framework, proving almost-sure convergence to the minimum-distance solution $x^\dagger$ and a regularizing property for noisy data under an a priori stopping rule. It derives convergence rates under a conditional stability assumption and provides a detailed convergence analysis for exact and noisy data, including descent in the Bregman distance and staying within a region where forward maps are well behaved. Numerical experiments on Schlieren tomography and electrical impedance tomography illustrate the practical benefits of Banach-space formulations, including sparsity promotion and robustness to non-Gaussian noise. The work extends the theory of SGD in Banach spaces for nonlinear inverse problems, bridging stochastic optimization, regularization, and PDE-based imaging.

Abstract

Stochastic gradient descent (SGD) and its variants are widely used and highly effective optimization methods in machine learning, especially for neural network training. By using a single datum or a small subset of the data, selected randomly at each iteration, SGD scales well to problem size and has been shown to be effective for solving large-scale inverse problems. In this work, we investigate SGD for solving nonlinear inverse problems in Banach spaces through the lens of iterative regularization. Under general assumptions, we prove almost sure convergence of the iterates to the minimum distance solution and show the regularizing property in expectation under an a priori stopping rule. Further, we establish convergence rates under the conditional stability assumptions for both exact and noisy data. Numerical experiments on Schlieren tomography and electrical impedance tomography are presented to show distinct features of the method.

Figures (8)

• Figure 1: The noisy data used in the recovery: (b)-(c) Gaussian noise; (d): salt-and-pepper noise.
• Figure 2: The objective $\Psi$ with respect to the batch size $b$. In the top row we use noise level $\epsilon = 5\times 10^{-2}$ and in the bottom row $\epsilon = 1\times 10^{-2}$.
• Figure 3: The best reconstruction errors $e({\boldsymbol{x}})$ with respect to the noise level for three choices ${\cal X}=L^{r_{\cal X}}$.
• Figure 4: The reconstruction results with batch size $b=18$. (a)-(c): $\epsilon = 5\times 10^{-2}$. (d)-(f): $\epsilon = 1\times 10^{-2}$.
• Figure 5: The reconstructions of the phantom from the data in Fig. \ref{['fig:ST_data']}(d), by SGD with $b=30$.

Theorems & Definitions (41)

• Definition 1
• Definition 2: Duality map
• Theorem 1: TBHK_12
• Definition 3
• Theorem 2: TBHK_12
• Lemma 1: Coercivity of the Bregman distance JinKereta:2023
• Remark 1
• Remark 2
• Lemma 2
• proof