Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Momentum accelerated power iterations and the restarted Lanczos method

Alessandro Barletta, Nicholas Marshall, Sara Pollock

TL;DR

The paper analyzes two principal approaches to computing extremal eigenpairs of large Hermitian matrices: restarted Lanczos and momentum accelerated power iterations. Using Chebyshev polynomial theory, it derives rate-based comparisons and shows momentum methods dominate when the spectral gap is small (i.e., $| frac{\lambda_2}{\lambda_1}|\to1$); it also demonstrates that employing momentum as a preconditioner markedly accelerates restarted Lanczos. Numerical experiments on benchmark sets confirm the theoretical findings and illustrate substantial improvements in iteration counts and residual accuracy, with momentum preconditioning often yielding the fastest convergence. The work suggests a practical, generalizable momentum-preconditioned Krylov framework and points toward extending the ideas to nonsymmetric problems via Arnoldi and generalized momentum methods.

Abstract

In this paper we compare two methods for finding extremal eigenvalues and eigenvectors: the restarted Lanczos method and momentum accelerated power iterations. The convergence of both methods is based on ratios of Chebyshev polynomials evaluated at subdominant and dominant eigenvalues; however, the convergence is not the same. Here we compare the theoretical convergence properties of both methods, and determine the relative regimes where each is more efficient. We further introduce a preconditioning technique for the restarted Lanczos method using momentum accelerated power iterations, and demonstrate its effectiveness. The theoretical results are backed up by numerical tests on benchmark problems.

Momentum accelerated power iterations and the restarted Lanczos method

TL;DR

The paper analyzes two principal approaches to computing extremal eigenpairs of large Hermitian matrices: restarted Lanczos and momentum accelerated power iterations. Using Chebyshev polynomial theory, it derives rate-based comparisons and shows momentum methods dominate when the spectral gap is small (i.e., ); it also demonstrates that employing momentum as a preconditioner markedly accelerates restarted Lanczos. Numerical experiments on benchmark sets confirm the theoretical findings and illustrate substantial improvements in iteration counts and residual accuracy, with momentum preconditioning often yielding the fastest convergence. The work suggests a practical, generalizable momentum-preconditioned Krylov framework and points toward extending the ideas to nonsymmetric problems via Arnoldi and generalized momentum methods.

Abstract

In this paper we compare two methods for finding extremal eigenvalues and eigenvectors: the restarted Lanczos method and momentum accelerated power iterations. The convergence of both methods is based on ratios of Chebyshev polynomials evaluated at subdominant and dominant eigenvalues; however, the convergence is not the same. Here we compare the theoretical convergence properties of both methods, and determine the relative regimes where each is more efficient. We further introduce a preconditioning technique for the restarted Lanczos method using momentum accelerated power iterations, and demonstrate its effectiveness. The theoretical results are backed up by numerical tests on benchmark problems.

Paper Structure

This paper contains 19 sections, 3 theorems, 41 equations, 5 figures, 6 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Notation
  3. Algorithms
  4. Background
  5. Chebyshev polynomials
  6. Momentum accelerated power iteration
  7. Normalized iteration
  8. Convergence of each eigenmode
  9. Dynamic implementation
  10. (Restarted) Lanczos
  11. Comparing rates
  12. Example 1
  13. Example 2
  14. Preconditioning
  15. Numerical examples
  16. ...and 4 more sections

Key Result

Lemma 2.1

APZ24 Let $\rho \in (0,1)$ and consider $\varepsilon$ small enough so that $(2 \rho \varepsilon + \varepsilon^2)/(1+\rho^2) < 1$. Let $\rho_k = \rho + \varepsilon$ and define $r_{k+1} = 2 \rho_k/(1 + \rho_k^2)$, as in iteration eqn:mpscheme and algorithm alg:dymo. Then

Figures (5)

  • Figure 1: Residual convergence of restarted Lanczos($m$) algorithm \ref{['alg:lanczosm']} and the dynamic momentum algorithm \ref{['alg:dymo']} for example 1, together with reference lines showing the predicted rates, with $n = 1024$ and $m=8$ (top left), $m=16$ (top right), $m=32$ (bottom left), $m=64$ (bottom right).
  • Figure 2: Average reduction of subdominant eigenmodes by eigenvalue for the power, restarted Lanzcos($m$), and dynamic momentum algorithms applied to example 1 in subsection \ref{['subsec:ex1']}. Left: $m = 16$; right: $m=32$. The average convergence rate for each mode was computed as the slope of the regression line found by Matlab's polyfit function.
  • Figure 3: Average reduction of subdominant eigenmodes by eigenvalue for the power, restarted Lanzcos($m$), and dynamic momentum algorithms. Left: $m = 64$; right: $m=64$ (detail). The average convergence rate for each mode was computed as the slope of the regression line found by Matlab's polyfit function.
  • Figure 4: Reduction of subdominant eigenmodes by matrix-vector multiplies $k$ for the power, restarted Lanzcos($m$), dynamic momentum, and preconditioned restarted-Lanczos($m$) algorithms for example 1 of subsection \ref{['subsec:ex1']} with $n = 1024$. Left: mode $j=12$ with $\lambda_j/\lambda_1 = 1013/1024 \approx 0.9893$; right: mode $j=64$ with $\lambda_j/\lambda_1 = 961/1024 \approx 0.9385$
  • Figure 5: Residual convergence of restarted Lanczos($m$) algorithm \ref{['alg:lanczosm']}, the dynamic momentum algorithm \ref{['alg:dymo']}, the momentum preconditioned Lanczos($m$) algorithm \ref{['alg:mPL']} denoted Lan(mPx,A), and the power preconditioned Lanczos($m$) algorithm denoted Lan(pPx,A), for example 2 of subsection \ref{['subsec:ex2']}, with $n = 2048$ and $m=64$ (left), and $m=128$ (right).

Theorems & Definitions (7)

  • Lemma 2.1
  • Lemma 2.2
  • proof
  • Remark 2.1
  • Theorem 2.1
  • Remark 3.1
  • Remark 3.2