Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Average block nonlinear Kaczmarz methods with adaptive momentum for nonlinear systems of equations

Renjie Ding, Dongling Wang, Jun Zou

Abstract

The Kaczmarz method is widely recognized as an efficient iterative algorithm for solving large-scale linear systems, owing to its simplicity and low memory requirements. However, the development of its nonlinear extensions for solving large-scale nonlinear systems has seen limited progress. In this work, we introduce a new family of momentum-accelerated averaging block nonlinear Kaczmarz methods tailored for large-scale nonlinear systems and ill-posed problems. Our contributions are twofold: (1) We develop an adaptive strategy for selecting step sizes and momentum coefficients, leading to a new average block nonlinear Kaczmarz method with adaptive momentum (ABNKAm). This algorithm achieves high computational efficiency by requiring only minimal inner-product computations per iteration, which significantly reduces both arithmetic complexity and memory usage. (2) We establish rigorous convergence of the ABNKAm under mild assumptions, proving that the method converges exponentially to the unique solution nearest to the initial point. Moreover, under suitable conditions, we provide a theoretical justification of acceleration of the proposed ABNKAm with momentum. Extensive numerical experiments demonstrate that ABNKAm outperforms existing nonlinear Kaczmarz variants in terms of both iteration count and computational time, with particularly notable gains in large-scale problems.

Average block nonlinear Kaczmarz methods with adaptive momentum for nonlinear systems of equations

Abstract

The Kaczmarz method is widely recognized as an efficient iterative algorithm for solving large-scale linear systems, owing to its simplicity and low memory requirements. However, the development of its nonlinear extensions for solving large-scale nonlinear systems has seen limited progress. In this work, we introduce a new family of momentum-accelerated averaging block nonlinear Kaczmarz methods tailored for large-scale nonlinear systems and ill-posed problems. Our contributions are twofold: (1) We develop an adaptive strategy for selecting step sizes and momentum coefficients, leading to a new average block nonlinear Kaczmarz method with adaptive momentum (ABNKAm). This algorithm achieves high computational efficiency by requiring only minimal inner-product computations per iteration, which significantly reduces both arithmetic complexity and memory usage. (2) We establish rigorous convergence of the ABNKAm under mild assumptions, proving that the method converges exponentially to the unique solution nearest to the initial point. Moreover, under suitable conditions, we provide a theoretical justification of acceleration of the proposed ABNKAm with momentum. Extensive numerical experiments demonstrate that ABNKAm outperforms existing nonlinear Kaczmarz variants in terms of both iteration count and computational time, with particularly notable gains in large-scale problems.

Paper Structure

This paper contains 9 sections, 12 theorems, 69 equations, 4 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. The average block nonlinear Kaczmarz method with momentum
  3. General framework
  4. The average block nonlinear Kaczmarz method with adaptive momentum
  5. Convergence analysis
  6. Convergence analysis of Algorithm \ref{['alg:ABNKAm']}
  7. Convergence analysis of Algorithm \ref{['alg:ABNKAm2']}
  8. Numerical experiments
  9. Concluding remarks

Key Result

Lemma 3.2

hanke1995convergencehaltmeier2007kaczmarzkaltenbacher2008iterativetan2024nonlinearxiao2024averaging Let $\mathcal{J} \subseteq \{1, 2, \ldots, m\}$. Under Assumption as:asc(ii)-(iii), it holds for all $x,\widetilde{x}\in B_{\delta_0}(x_0)$ that

Figures (4)

  • Figure 1: Curves of ERROR and RES against IT for $m=1000$, Modified Rosenbrock Problem
  • Figure 2: Chandrasekhar H-equation ($m=10^4$). Left: Curve of error versus IT; Right; Curve of momentum parameter versus IT.
  • Figure 3: Extended Powell badly scaled problem with $m=10^3$: Curvers of RES against IT and CPU for different methods
  • Figure 4: Extended Powell badly scaled problem with $m=1\cdot 10^3$: Curvers of errors against IT and CPU for different methods

Theorems & Definitions (21)

  • Remark 2.1
  • Remark 2.2
  • Remark 2.3
  • Remark 2.4
  • Lemma 3.2
  • Lemma 3.3
  • Lemma 3.4
  • Lemma 3.5
  • Lemma 3.6
  • Corollary 3.7
  • ...and 11 more