Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On convergence of greedy block nonlinear Kaczmarz methods with momentum

Naiyu Jiang, Wendi Bao, Lili Xing, Weiguo Li

TL;DR

The paper tackles solving nonlinear systems $f(x)=0$ by introducing two momentum-augmented, pseudoinverse-free greedy block nonlinear Kaczmarz methods, RBWNK-m and MRWNK-m, built from RBWNK/MRWNK and the heavy-ball momentum. It develops two greedy block index rules and proves convergence under a local tangential cone condition without assuming full column rank of the Jacobian, providing explicit rate bounds dependent on momentum and selection parameters. The theoretical results are complemented by numerical experiments on several nonlinear test problems, where the momentum-augmented schemes consistently reduce iterations and CPU time compared to their non-momentum counterparts. The work demonstrates that incorporating momentum yields meaningful improvements in convergence speed for nonlinear Kaczmarz-type methods with greedy block strategies.

Abstract

In this paper, for solving nonlinear systems we propose two pseudoinverse-free greedy block methods with momentum by combining the residual-based weighted nonlinear Kaczmarz and heavy ball methods. Without the full column rank assumptions on Jacobi matrices of nonlinear systems, we provide a thorough convergence analysis, and derive upper bounds for the convergence rates of the new methods. Numerical experiments demonstrate that the proposed methods with momentum are much more effective than the existing ones.

On convergence of greedy block nonlinear Kaczmarz methods with momentum

TL;DR

The paper tackles solving nonlinear systems by introducing two momentum-augmented, pseudoinverse-free greedy block nonlinear Kaczmarz methods, RBWNK-m and MRWNK-m, built from RBWNK/MRWNK and the heavy-ball momentum. It develops two greedy block index rules and proves convergence under a local tangential cone condition without assuming full column rank of the Jacobian, providing explicit rate bounds dependent on momentum and selection parameters. The theoretical results are complemented by numerical experiments on several nonlinear test problems, where the momentum-augmented schemes consistently reduce iterations and CPU time compared to their non-momentum counterparts. The work demonstrates that incorporating momentum yields meaningful improvements in convergence speed for nonlinear Kaczmarz-type methods with greedy block strategies.

Abstract

In this paper, for solving nonlinear systems we propose two pseudoinverse-free greedy block methods with momentum by combining the residual-based weighted nonlinear Kaczmarz and heavy ball methods. Without the full column rank assumptions on Jacobi matrices of nonlinear systems, we provide a thorough convergence analysis, and derive upper bounds for the convergence rates of the new methods. Numerical experiments demonstrate that the proposed methods with momentum are much more effective than the existing ones.

Paper Structure

This paper contains 11 sections, 7 theorems, 49 equations, 5 figures, 6 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Notations and Preliminaries
  3. The residual-based weighted nonlinear Kaczmarz method
  4. The heavy ball method
  5. Main Results
  6. Greedy block Kaczmarz methods with momentum
  7. Convergence analysis
  8. Numerical experiments
  9. Conclusion
  10. The proof of E in Lemma \ref{['lem']}
  11. The proof of . in Lemma \ref{['lem']}

Key Result

Lemma 3.1

Xing25 If $f(x):\mathcal{D}\subseteq R^n\to R^m$ satisfies the local tangential cone condition in $\mathcal{D}$ and $x_*$ is a solution of (eq1.1) in $\mathcal{D}$. Then the following statements hold.

Figures (5)

  • Figure 4.1 : IT of the four methods versus $q$ for Singular Broyden problem with $n=100$, $500$, $1000$
  • Figure 4.2 : The results of the Singular Broyden problem $n=100$, $500$, $1000$
  • Figure 4.3 : IT of the four methods versus $q$ for H-equation with $m=n=100$, $500$, $1000$
  • Figure 4.4 : The results of the H-equation with $m=n=100$, $500$, $1000$
  • Figure 4.5 : IT of the four methods versus $q$ for nondquar problem with $m=n=200$, $400$, $800$

Theorems & Definitions (18)

  • Remark 2.1
  • Definition 3.1
  • Lemma 3.1
  • proof
  • Lemma 3.2
  • proof
  • Lemma 3.3
  • Lemma 3.4
  • Lemma 3.5
  • proof
  • ...and 8 more