Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Stochastic Iteratively Regularized Gauss-Newton Method

El Houcine Bergou, Neil K. Chada, Youssef Diouane

TL;DR

This work focuses on developing and motivating a stochastic version of a wellknown inverse problem methodology, the iteratively regularized Gauss–Newton method, by introducing a new algorithm, the stochastic iteratively regularized Gauss–Newton method (SIRGNM).

Abstract

This work focuses on developing and motivating a stochastic version of a wellknown inverse problem methodology. Specifically, we consider the iteratively regularized Gauss-Newton method, originally proposed by Bakushinskii for infinite-dimensional problems. Recent work have extended this method to handle sequential observations, rather than a single instance of the data, demonstrating notable improvements in reconstruction accuracy. In this paper, we further extend these methods to a stochastic framework through mini-batching, introducing a new algorithm, the stochastic iteratively regularized Gauss-Newton method (SIRGNM). Our algorithm is designed through the use randomized sketching. We provide an analysis for the SIRGNM, which includes a preliminary error decomposition and a convergence analysis, related to the residuals. We provide numerical experiments on a 2D elliptic PDE example. This illustrates the effectiveness of the SIRGNM, through maintaining a similar level of accuracy while reducing on the computational time.

A Stochastic Iteratively Regularized Gauss-Newton Method

TL;DR

This work focuses on developing and motivating a stochastic version of a wellknown inverse problem methodology, the iteratively regularized Gauss–Newton method, by introducing a new algorithm, the stochastic iteratively regularized Gauss–Newton method (SIRGNM).

Abstract

This work focuses on developing and motivating a stochastic version of a wellknown inverse problem methodology. Specifically, we consider the iteratively regularized Gauss-Newton method, originally proposed by Bakushinskii for infinite-dimensional problems. Recent work have extended this method to handle sequential observations, rather than a single instance of the data, demonstrating notable improvements in reconstruction accuracy. In this paper, we further extend these methods to a stochastic framework through mini-batching, introducing a new algorithm, the stochastic iteratively regularized Gauss-Newton method (SIRGNM). Our algorithm is designed through the use randomized sketching. We provide an analysis for the SIRGNM, which includes a preliminary error decomposition and a convergence analysis, related to the residuals. We provide numerical experiments on a 2D elliptic PDE example. This illustrates the effectiveness of the SIRGNM, through maintaining a similar level of accuracy while reducing on the computational time.
Paper Structure (18 sections, 6 theorems, 59 equations, 4 figures, 1 table, 4 algorithms)

This paper contains 18 sections, 6 theorems, 59 equations, 4 figures, 1 table, 4 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Outline
  4. Background material
  5. Stochastic IRGNM
  6. Towards a general stochastic framework for IRGNM
  7. On the randomized sketching operator
  8. Discussion on Assumption \ref{['asm:1']}
  9. Convergence Analysis
  10. Preliminary error decomposition
  11. Numerical Experiments
  12. Overview of algorithms
  13. Darcy flow model
  14. Parameter choices
  15. Summary from figures
  16. ...and 3 more sections

Key Result

Lemma 4.1

Let Assumption asm:1 hold. Then the randomized error $\widetilde{\mathrm{err}}_n$ evaluated at $F_{n}^{\dagger}$ is an unbiased estimator of ${\mathrm{err}_n}$ at $F_{n}^{\dagger}$, i.e.,

Figures (4)

  • Figure 1: Stochastic IRGNM and dIRGNM for a discontinuous truth $u^{\dagger}$.
  • Figure 2: IRGNM and SIRGNM with constant regularization term.
  • Figure 3: Final reconstruction plots of various methods on the discontinuous groundtruth.
  • Figure 4: Final reconstruction plots of various methods on the smooth groundtruth.

Theorems & Definitions (13)

  • Remark 2.1
  • Lemma 4.1
  • Lemma 4.2
  • proof
  • Remark 4.3
  • Lemma 4.4: Preliminary error estimate
  • proof
  • Lemma 4.5
  • proof
  • Theorem 4.6
  • ...and 3 more