Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On inertial Levenberg-Marquardt type methods for solving nonlinear ill-posed operator equations

Antonio Leitão, Joel C. Rabelo, Dirk A. Lorenz, Maximilian Winkler

TL;DR

This work introduces the inertial Levenberg-Marquardt (inLM) method for nonlinear ill-posed operator equations $F(x)=y$ by incorporating an extrapolated search point $w_k = x_k + \alpha_k(x_k - x_{k-1})$ and a LM-type update around $w_k$. Under suitable assumptions, the authors prove monotonicity and strong convergence for exact data, and establish regularization properties with a discrepancy principle for noisy data, including stability and semi-convergence results. Numerical experiments on elliptic PDE parameter identification and an inverse NN training problem demonstrate faster convergence and improved accuracy of inLM compared to canonical LM, highlighting its practical effectiveness as a robust regularization tool for nonlinear inverse problems. The results suggest that inLM can be a valuable, computationally efficient alternative for solving a broad class of ill-posed nonlinear equations in applied mathematics.

Abstract

In these notes we propose and analyze an inertial type method for obtaining stable approximate solutions to nonlinear ill-posed operator equations. The method is based on the Levenberg-Marquardt (LM) iteration. The main obtained results are: monotonicity and convergence for exact data, stability and semi-convergence for noisy data. Regarding numerical experiments we consider: i) a parameter identification problem in elliptic PDEs, ii) a parameter identification problem in machine learning; the computational efficiency of the proposed method is compared with canonical implementations of the LM method.

On inertial Levenberg-Marquardt type methods for solving nonlinear ill-posed operator equations

TL;DR

This work introduces the inertial Levenberg-Marquardt (inLM) method for nonlinear ill-posed operator equations by incorporating an extrapolated search point and a LM-type update around . Under suitable assumptions, the authors prove monotonicity and strong convergence for exact data, and establish regularization properties with a discrepancy principle for noisy data, including stability and semi-convergence results. Numerical experiments on elliptic PDE parameter identification and an inverse NN training problem demonstrate faster convergence and improved accuracy of inLM compared to canonical LM, highlighting its practical effectiveness as a robust regularization tool for nonlinear inverse problems. The results suggest that inLM can be a valuable, computationally efficient alternative for solving a broad class of ill-posed nonlinear equations in applied mathematics.

Abstract

In these notes we propose and analyze an inertial type method for obtaining stable approximate solutions to nonlinear ill-posed operator equations. The method is based on the Levenberg-Marquardt (LM) iteration. The main obtained results are: monotonicity and convergence for exact data, stability and semi-convergence for noisy data. Regarding numerical experiments we consider: i) a parameter identification problem in elliptic PDEs, ii) a parameter identification problem in machine learning; the computational efficiency of the proposed method is compared with canonical implementations of the LM method.
Paper Structure (16 sections, 11 theorems, 72 equations, 10 figures, 2 algorithms)

This paper contains 16 sections, 11 theorems, 72 equations, 10 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. State of the art
  3. The Levenberg-Marquardt (LM) method
  4. Inertial iterative methods
  5. Contribution
  6. Outline
  7. The inertial Levenberg-Marquardt method
  8. Description of method
  9. A strong convergence result
  10. Regularization properties
  11. Numerical experiments
  12. Parameter identification in an elliptic PDE
  13. An inverse problem in neural network training
  14. Formulation of the inverse problem.
  15. Numerical implementations.
  16. ...and 1 more sections

Key Result

Lemma 2.2

Let (A1) hold and $(x_k)$, $(w_k)$ be sequences generated by Algorithm alg:init-exact. Thus for $x \in X$.

Figures (10)

  • Figure 1: Left to right: exact PDE solution $u^\dagger$ (exact solution to the forward problem), $c^\dagger$ (exact solution to the inverse problem), reconstruction of $c^\dagger$ by \ref{['eqn:c_from_simple_division']}
  • Figure 2: Results without noise: errors for $\alpha_k\equiv\alpha\in \{0,0.2,...,1\}$ from red to blue color. Left: distances $\|c_k-c^\dagger\|_2$, right: residuals $\|F(c_k)-u^\dagger\|_2$
  • Figure 3: Results without noise, left to right: iterates $w_{10}$ for $\alpha=0$ (non-accelerated Levenberg-Marquardt method), $\alpha=0.6$ and $\alpha=1$
  • Figure 4: Results without noise, left to right: iterates $w_{500}$ for $\alpha=0$ (non-accelerated Levenberg-Marquardt method), $\alpha=0.6$ and $\alpha=1$
  • Figure 5: Left: noisy PDE solution $u^\delta$, right: naive approximation of $c^\dagger$ by \ref{['eqn:c_from_simple_division']}
  • ...and 5 more figures

Theorems & Definitions (26)

  • Remark 2.1: Comments on Algorithm \ref{['alg:init-exact']}
  • Lemma 2.2
  • proof
  • Lemma 2.3
  • proof
  • Remark 2.5
  • Proposition 2.6
  • proof
  • Proposition 2.7
  • proof
  • ...and 16 more