Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Polyak momentum variants of the greedy deterministic single and multiple row-action methods

Nian-Ci Wu, Qian Zuo, Yatian Wang

TL;DR

This work blends greedy deterministic row-action strategies with Polyak momentum to accelerate Kaczmarz-type solvers for consistent linear systems. By introducing the mMWRK and mFDBK variants, the authors prove convergence to the minimum-norm least-squares solution and establish linear rates under suitable choices of the step-size $\alpha$ and momentum $\beta$. Extensive numerical experiments on synthetic data and curve-fitting problems demonstrate that momentum variants achieve faster convergence and notable iteration-count reductions compared to their non-momentum counterparts. The results highlight the practical potential of deterministic greedy-row methods augmented with Polyak momentum for large-scale linear-systems tasks relevant to applications such as computer-aided geometric design.

Abstract

For solving a consistent system of linear equations, the classical row-action (also known as Kaczmarz) method is a simple while really effective iteration solver. Based on the greedy index selection strategy and Polyak's heavy-ball momentum acceleration technique, we propose two deterministic row-action methods and establish the corresponding convergence theory. We show that our algorithm can linearly converge to a least-squares solution with minimum Euclidean norm. Several numerical studies have been presented to corroborate our theoretical findings. Real-world applications, such as data fitting in computer-aided geometry design, are also presented for illustrative purposes.

On the Polyak momentum variants of the greedy deterministic single and multiple row-action methods

TL;DR

This work blends greedy deterministic row-action strategies with Polyak momentum to accelerate Kaczmarz-type solvers for consistent linear systems. By introducing the mMWRK and mFDBK variants, the authors prove convergence to the minimum-norm least-squares solution and establish linear rates under suitable choices of the step-size and momentum . Extensive numerical experiments on synthetic data and curve-fitting problems demonstrate that momentum variants achieve faster convergence and notable iteration-count reductions compared to their non-momentum counterparts. The results highlight the practical potential of deterministic greedy-row methods augmented with Polyak momentum for large-scale linear-systems tasks relevant to applications such as computer-aided geometric design.

Abstract

For solving a consistent system of linear equations, the classical row-action (also known as Kaczmarz) method is a simple while really effective iteration solver. Based on the greedy index selection strategy and Polyak's heavy-ball momentum acceleration technique, we propose two deterministic row-action methods and establish the corresponding convergence theory. We show that our algorithm can linearly converge to a least-squares solution with minimum Euclidean norm. Several numerical studies have been presented to corroborate our theoretical findings. Real-world applications, such as data fitting in computer-aided geometry design, are also presented for illustrative purposes.
Paper Structure (13 sections, 5 theorems, 53 equations, 8 figures, 6 tables, 4 algorithms)

This paper contains 13 sections, 5 theorems, 53 equations, 8 figures, 6 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. The greedy deterministic row-action methods
  3. The MWRK method
  4. The FDBK method
  5. The proposed methods
  6. The mMWRK method
  7. The mFDBK method
  8. Convergence analyses
  9. Numerical simulations
  10. Choice of $\alpha$ and $\beta$
  11. Synthetic data
  12. Real-world application: curve fitting
  13. Conclusions

Key Result

Theorem 2.1

19DG Let $A\in \mathbb{C}^{m\times n}$ be a matrix without any zero rows and $b \in \mathbb{C}^{m}$. The iteration sequence $\left\{x^{(k )}\right\}_{k=0}^{\infty}$, generated by the MWRK method starting from any initial guess $x^{(0)} \in {\mathcal{R}}(A^\ast)$, exists and converges to the unique l for $k=0,1,2,\cdots$, where the constant $\widetilde{\gamma} = \max_{i\in [m]}\left\{ \sum_{j=1,j\n

Figures (8)

  • Figure 1: Geometric interpretation of the mMWRK (a) and MWRK (b) 19DG77McCormick iterations.
  • Figure 2: The number of iteration steps of mMWRK (left) and mFDBK (right) with different $(\alpha,\beta)$ for solving a consistent linear systems, where the test matrix is generated by randn$(m,n)$.
  • Figure 3: The number of iteration steps of mMWRK (left) and mFDBK (right) with different $(\alpha,\beta)$ for solving a consistent linear systems, where the test matrix is generated by randn$(m,n)$.
  • Figure 4: RSE versus IT obtained by mMWRK (a) and mFDBK (b) for Example \ref{['ERMR:example1']} when $m=15000$, $n=350$, $r = n/10$, and $\kappa = n/10$.
  • Figure 5: RSE versus IT obtained by mMWRK (a) and mFDBK (b) for Example \ref{['ERMR:example1']} when $m=350$, $n=15000$, $r = m/10$, and $\kappa = m/10$.
  • ...and 3 more figures

Theorems & Definitions (14)

  • Theorem 2.1
  • Theorem 2.2
  • Remark 3.1
  • Remark 3.2
  • Remark 3.3
  • Remark 3.4
  • Remark 3.5
  • Lemma 4.1
  • Theorem 4.1
  • Remark 4.1
  • ...and 4 more