Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastic variance reduced gradient method for linear ill-posed inverse problems

Qinian Jin, Liuhong Chen

TL;DR

The authors extend the stochastic variance reduced gradient (SVRG) framework to large-scale linear ill-posed inverse problems in Hilbert spaces, introducing a two-step-update scheme with independent step-sizes to stabilize iterations under noise. They prove convergence with an a priori stopping rule under a benchmark source condition and, using a perturbation argument, establish convergence without any source condition; a discrepancy-principle variant yields finite, almost-sure termination. The paper also provides a comprehensive numerical study on Fredholm first-kind problems, showing that SVRG (especially with a reduced update frequency) can outperform the classical Landweber method in large-scale settings while delivering reliable reconstructions. These results offer a scalable, variance-reduced alternative for iterative regularization in ill-posed problems and support practical use with a posteriori stopping. The findings pave the way for adaptive step-size strategies, broader priors, and nonlinear extensions.

Abstract

In this paper we apply the stochastic variance reduced gradient (SVRG) method, which is a popular variance reduction method in optimization for accelerating the stochastic gradient method, to solve large scale linear ill-posed systems in Hilbert spaces. Under {\it a priori} choices of stopping indices, we derive a convergence rate result when the sought solution satisfies a benchmark source condition and establish a convergence result without using any source condition. To terminate the method in an {\it a posteriori} manner, we consider the discrepancy principle and show that it terminates the method in finite many iteration steps almost surely. Various numerical results are reported to test the performance of the method.

Stochastic variance reduced gradient method for linear ill-posed inverse problems

TL;DR

The authors extend the stochastic variance reduced gradient (SVRG) framework to large-scale linear ill-posed inverse problems in Hilbert spaces, introducing a two-step-update scheme with independent step-sizes to stabilize iterations under noise. They prove convergence with an a priori stopping rule under a benchmark source condition and, using a perturbation argument, establish convergence without any source condition; a discrepancy-principle variant yields finite, almost-sure termination. The paper also provides a comprehensive numerical study on Fredholm first-kind problems, showing that SVRG (especially with a reduced update frequency) can outperform the classical Landweber method in large-scale settings while delivering reliable reconstructions. These results offer a scalable, variance-reduced alternative for iterative regularization in ill-posed problems and support practical use with a posteriori stopping. The findings pave the way for adaptive step-size strategies, broader priors, and nonlinear extensions.

Abstract

In this paper we apply the stochastic variance reduced gradient (SVRG) method, which is a popular variance reduction method in optimization for accelerating the stochastic gradient method, to solve large scale linear ill-posed systems in Hilbert spaces. Under {\it a priori} choices of stopping indices, we derive a convergence rate result when the sought solution satisfies a benchmark source condition and establish a convergence result without using any source condition. To terminate the method in an {\it a posteriori} manner, we consider the discrepancy principle and show that it terminates the method in finite many iteration steps almost surely. Various numerical results are reported to test the performance of the method.
Paper Structure (7 sections, 10 theorems, 100 equations, 3 figures, 3 tables, 3 algorithms)

This paper contains 7 sections, 10 theorems, 100 equations, 3 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Stability estimate
  3. Rate of convergence
  4. Convergence
  5. The discrepancy principle
  6. Numerical simulations
  7. Conclusion

Key Result

Lemma 2.1

Let $L$ be defined by (L). If $\gamma_0>0$ and $\gamma_1>0$ are chosen such that then for all integers $n \ge 0$, where

Figures (3)

  • Figure 1: Reconstruction error of SVRG using various parameters of $\alpha, \beta$ and the relative noise level $\delta_{rel}$. The rows from top to bottom refer to phillips, gravity and shaw, respectively.
  • Figure 2: Boxplots of the relative error $||x_{n_{\delta}}^{\delta}-x^{\dagger}||^{2}/||x^{\dagger}||^{2}$ and the stopping index $n_\delta$ for the model probelms with $N=10000$. The rows from top to bottom refer to phillips, gravity and shaw respectively.
  • Figure 3: The sought solution $x^{\dagger}$ and the reconstruction results by SVRG using noisy data with various relative noise levels. The rows from top to bottom refer to phillips, gravity and shaw respectively.

Theorems & Definitions (23)

  • Lemma 2.1
  • proof
  • Remark 2.1
  • Lemma 3.1
  • proof
  • Lemma 3.2
  • proof
  • Lemma 3.3
  • proof
  • Theorem 3.4
  • ...and 13 more