Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastic gradient descent method with convex penalty for ill-posed problems in Banach spaces

Ruixue Gu, Zhenwu Fu, Bo Han, Hongsun Fu

Abstract

In this work, we investigate a stochastic gradient descent method for solving inverse problems that can be written as systems of linear or nonlinear ill-posed equations in Banach spaces. The method uses only a randomly selected equation at each iteration and employs the convex function as the penalty term, and thus it is scalable to the problem size and has the ability to detect special features of solutions such as nonnegativity and piecewise constancy. To suppress the oscillation in iterates and reduce the semi-convergence of such methods, by incorporating the spirit of discrepancy principle, an adaptive strategy for choosing the step size is suggested. Under certain conditions, we establish the regularization results of the method under an {\it a priori} stopping rule. Several numerical simulations on computed tomography and schlieren imaging are provided to demonstrate the effectiveness of the method. Finally, we study an {\it a posteriori} stopping rule for SGD-$θ$ method and show the finite iterations termination property.

Stochastic gradient descent method with convex penalty for ill-posed problems in Banach spaces

Abstract

In this work, we investigate a stochastic gradient descent method for solving inverse problems that can be written as systems of linear or nonlinear ill-posed equations in Banach spaces. The method uses only a randomly selected equation at each iteration and employs the convex function as the penalty term, and thus it is scalable to the problem size and has the ability to detect special features of solutions such as nonnegativity and piecewise constancy. To suppress the oscillation in iterates and reduce the semi-convergence of such methods, by incorporating the spirit of discrepancy principle, an adaptive strategy for choosing the step size is suggested. Under certain conditions, we establish the regularization results of the method under an {\it a priori} stopping rule. Several numerical simulations on computed tomography and schlieren imaging are provided to demonstrate the effectiveness of the method. Finally, we study an {\it a posteriori} stopping rule for SGD- method and show the finite iterations termination property.
Paper Structure (10 sections, 7 theorems, 144 equations, 6 figures, 1 table)

This paper contains 10 sections, 7 theorems, 144 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Convergence
  4. Convergence analysis for exact data
  5. Regularization property
  6. Numerical experiments
  7. Computed Tomography
  8. Schlieren Imaging
  9. Conclusion and outlook
  10. SGD method under a posteriori stopping rule

Key Result

Lemma 2.1

Let $f :\mathcal{X} \to \left( { - \infty ,\infty } \right]$ be a proper, lower semi-continuous and $p$-convex function with $p\ge 2$ in the sense that (pconvex) is satisfied for some constant $\sigma>0$. Then,

Figures (6)

  • Figure 1: Computational results for CT under Gaussian noise.
  • Figure 2: Results for CT under Gaussian noise with ${\delta _{rel}}=0.01$.
  • Figure 3: Computational results for CT under salt-and-pepper noise. Left column: $\kappa=5\%$; Right column: $\kappa=10\%$.
  • Figure 4: Reconstruction results for CT by SGD-$\theta$ method with $r=1.1$ (left column), $r=1.5$ (middle column) and $r=2$ (right column) after 10000 iterations. Top row: salt-and-pepper noise with $\kappa=5\%$; Bottom row: salt-and-pepper noise with $\kappa=10\%$.
  • Figure 5: Computational results for Schlieren imaging under uniformly distributed noise.
  • ...and 1 more figures

Theorems & Definitions (19)

  • Lemma 2.1
  • Remark 3.1
  • Lemma 3.2
  • proof
  • Remark 3.3
  • Lemma 3.4
  • proof
  • Theorem 3.5
  • proof
  • Lemma 3.6
  • ...and 9 more