Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Applying acceleration to Krylov subspace eigenvalue solvers

Michelle Baker, Sara Pollock

Abstract

In this paper, we apply acceleration to the inverse-free preconditioned Krylov subspace method introduced by Golub and Ye, which solves the symmetric generalized eigenvalue problem for the algebraically smallest eigenvalue. As the method is an improvement on steepest descent, we consider acceleration based on Nesterov accelerated steepest descent and Polyak's heavy-ball method. We extend acceleration to the block version of the Krylov subspace method and prove convergence for a more generalized choice of subspace. We present numerical results demonstrating the effect of fixed and safeguarded-adaptive choice of the momentum parameter, which show convergence in fewer outer iterations compared with LOBPCG with the same subspace size and generally fewer iterations than the base method when solving for multiple clustered eigenvalues with small dimension size. We also provide an explanation for the acceleration seen from implementing Polyak's heavy-ball method, including justifying the given parameter range.

Applying acceleration to Krylov subspace eigenvalue solvers

Abstract

In this paper, we apply acceleration to the inverse-free preconditioned Krylov subspace method introduced by Golub and Ye, which solves the symmetric generalized eigenvalue problem for the algebraically smallest eigenvalue. As the method is an improvement on steepest descent, we consider acceleration based on Nesterov accelerated steepest descent and Polyak's heavy-ball method. We extend acceleration to the block version of the Krylov subspace method and prove convergence for a more generalized choice of subspace. We present numerical results demonstrating the effect of fixed and safeguarded-adaptive choice of the momentum parameter, which show convergence in fewer outer iterations compared with LOBPCG with the same subspace size and generally fewer iterations than the base method when solving for multiple clustered eigenvalues with small dimension size. We also provide an explanation for the acceleration seen from implementing Polyak's heavy-ball method, including justifying the given parameter range.
Paper Structure (24 sections, 2 theorems, 39 equations, 11 figures, 3 tables, 3 algorithms)

This paper contains 24 sections, 2 theorems, 39 equations, 11 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Inverse-free Krylov subspace method
  4. Block inverse-free Krylov subspace method
  5. Locally Optimal Preconditioned Conjugate Gradient method
  6. Example 1
  7. Example 2
  8. Acceleration
  9. Generalized Nesterov acceleration
  10. Generalized heavy-ball method
  11. Applying acceleration to the single vector method
  12. Analysis of extrapolation
  13. Numerical results
  14. Applying acceleration to the block vector method
  15. Analysis of convergence
  16. ...and 9 more sections

Key Result

Lemma 1

Let $\lambda_1\leq\dots\leq\lambda_n$ denote the eigenvalues of $(A,B)$. Let the Ritz values obtained from Algorithm 2.1 at the $k^{th}$ iteration be denoted by $\rho_1^{(k+1)}\leq\dots\leq\rho_b^{(k+1)}$ with corresponding Ritz vectors $x_1^{(k+1)},\dots,x_b^{(k+1)}$. Suppose further that The following relations hold

Figures (11)

  • Figure 1: Residual convergence solving for the smallest eigenvalue. Left and right differ by the randomly generated initial iterate.
  • Figure 2: The values and magnitudes of $\beta_k$ chosen by LOPCG for a randomly generated initial iterate.
  • Figure 3: Residual convergence solving for the smallest eigenvalue. Left: Good choice of initial iterate. Right: Bad choice of initial iterate.
  • Figure 5: Mesh for the barbell shaped domain
  • Figure 6: Residual convergence solving for the smallest eigenvalue. Left: $m=2$, right: $m=4$.
  • ...and 6 more figures

Theorems & Definitions (3)

  • Lemma 1
  • Theorem 1
  • proof