Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Levenberg-Marquardt method with Singular Scaling and applications

Everton Boos, Douglas S. Goncalves, Fermin S. V. Bazan

TL;DR

This work extends the Levenberg-Marquardt framework to singular scaling matrices, establishing well-definedness, local quadratic convergence under an error-bound condition, and global convergence to stationary points via line-search. By leveraging GSVD-based analysis, the authors bound step directions and show gradient-relatedness, enabling reliable global behavior even when residuals are nonzero due to noise. Applied to heat-conduction parameter identification, the method uses problem-tailored singular scaling operators to enforce smoothness, yielding markedly improved reconstructions of perfusion and conductivity compared with classic LMM, and demonstrating practical impact for inverse problems in PDE contexts.

Abstract

Inspired by certain regularization techniques for linear inverse problems, in this work we investigate the convergence properties of the Levenberg-Marquardt method using singular scaling matrices. Under a completeness condition, we show that the method is well-defined and establish its local quadratic convergence under an error bound assumption. We also prove that the search directions are gradient-related allowing us to show that limit points of the sequence generated by a line-search version of the method are stationary for the sum-of-squares function. The usefulness of the method is illustrated with some examples of parameter identification in heat conduction problems for which specific singular scaling matrices can be used to improve the quality of approximate solutions.

Levenberg-Marquardt method with Singular Scaling and applications

TL;DR

This work extends the Levenberg-Marquardt framework to singular scaling matrices, establishing well-definedness, local quadratic convergence under an error-bound condition, and global convergence to stationary points via line-search. By leveraging GSVD-based analysis, the authors bound step directions and show gradient-relatedness, enabling reliable global behavior even when residuals are nonzero due to noise. Applied to heat-conduction parameter identification, the method uses problem-tailored singular scaling operators to enforce smoothness, yielding markedly improved reconstructions of perfusion and conductivity compared with classic LMM, and demonstrating practical impact for inverse problems in PDE contexts.

Abstract

Inspired by certain regularization techniques for linear inverse problems, in this work we investigate the convergence properties of the Levenberg-Marquardt method using singular scaling matrices. Under a completeness condition, we show that the method is well-defined and establish its local quadratic convergence under an error bound assumption. We also prove that the search directions are gradient-related allowing us to show that limit points of the sequence generated by a line-search version of the method are stationary for the sum-of-squares function. The usefulness of the method is illustrated with some examples of parameter identification in heat conduction problems for which specific singular scaling matrices can be used to improve the quality of approximate solutions.
Paper Structure (11 sections, 10 theorems, 98 equations, 5 figures, 3 tables, 1 algorithm)

This paper contains 11 sections, 10 theorems, 98 equations, 5 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries and auxiliary results
  3. Local convergence
  4. Global convergence
  5. Application to parameter identification in heat conduction problems
  6. Reconstruction of 2D perfusion coefficient
  7. Reconstruction of thermal conductivity
  8. Example 1: isotropic conductivity
  9. Example 2: orthotropic conductivity
  10. Conclusion
  11. Some technical proofs

Key Result

Theorem 2.1

Consider a matrix pair $(A,L)$, where $A\in \mathbb{R}^{m\times n}$, $L\in \mathbb{R}^{p\times n}$, $m\geq n\geq p$, $\textup{rank} (L) = p$ and $\mathcal{N} (A) \cap \mathcal{N}(L) = \{ \textbf{0} \}$. Then, there exist matrices $U \in\mathbb{R}^{m\times n}$ and $V\in \mathbb{R}^{p\times p}$ with o where $\Sigma$ and $M$ are diagonal matrices: Moreover, the diagonal elements of $\Sigma$ and $M$

Figures (5)

  • Figure 1: Domain for perfusion estimation.
  • Figure 2: Relative error ${\rm RE}({\bf p}^{(j)})$ (left) and absolute temperature error (right) as functions of the iteration number, from data with ${\rm NL} = 0.001$. The blue dashed line corresponds to $L = I_n$ and the green dotted line to $L={\cal L}_2$. The horizontal red line on the right represents the value $\tau\|{\bf e}\|=0.0039.$
  • Figure 3: Exact and recovered 2D perfusion coefficient. Noisy data correspond to $\textup{NL} = 0.001$ (4 regularizers) and $\textup{NL} = 0.0001$ only for ${\cal L}_3$ (last plot from left to right).
  • Figure 4: Comparison between iterates generated by LMM and LMMSS.
  • Figure 5: Average results for $k_{11} (x,y)$ for some fixed $y$ values and $\textup{NL} = 0.001$. In the legends, $L_1$ and $L_2$ represent, respectively, $I_2 \otimes \tilde{\mathcal{L}}_1$ and $I_2 \otimes \tilde{\mathcal{L}}_2$.

Theorems & Definitions (18)

  • Theorem 2.1: GSVD, Hansen Hansen1998
  • Lemma 2.2
  • proof
  • Lemma 2.3
  • Lemma 3.1
  • proof
  • Lemma 3.2: Yamashita e Fukushima Yamashita
  • Lemma 3.3: Yamashita e Fukushima Yamashita
  • Theorem 3.4
  • proof
  • ...and 8 more