Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gradient-type subspace iteration methods for the symmetric eigenvalue problem

Foivos Alimisis, Yousef Saad, Bart Vandereycken

TL;DR

This work addresses computing the dominant invariant subspace of a real symmetric matrix $A$ by recasting the problem as optimization on the Grassmann manifold Gr$(n,p)$ of $p$-dimensional subspaces. It develops a gradient descent method and a Polak–Ribiere nonlinear conjugate gradient variant, both with an exact line search, and proves global convergence with local linear convergence in a neighborhood governed by the spectral gap $\delta$. The authors also provide practical implementation details to ensure accurate line searches and efficient matvecs, and they compare the methods against subspace iteration (with Chebyshev acceleration) and LOBCG across a suite of matrices, showing favorable performance in terms of matrix-vector products and runtime in many cases. These methods are particularly relevant for applications like SCF iterations in electronic structure calculations where spectrum information is not readily available and robustness to matrix changes is valuable.

Abstract

This paper explores variants of the subspace iteration algorithm for computing approximate invariant subspaces. The standard subspace iteration approach is revisited and new variants that exploit gradient-type techniques combined with a Grassmann manifold viewpoint are developed. A gradient method as well as a nonlinear conjugate gradient technique are described. Convergence of the gradient-based algorithm is analyzed and a few numerical experiments are reported, indicating that the proposed algorithms are sometimes superior to standard algorithms. This includes the Chebyshev-based subspace iteration and the locally optimal block conjugate gradient method, when compared in terms of number of matrix vector products and computational time, resp. The new methods, on the other hand, do not require estimating optimal parameters. An important contribution of this paper to achieve this good performance is the accurate and efficient implementation of an exact line search. In addition, new convergence proofs are presented for the non-accelerated gradient method that includes a locally exponential convergence if started in a $\mathcal{O(\sqrtδ)}$ neighbourhood of the dominant subspace with spectral gap $δ$.

Gradient-type subspace iteration methods for the symmetric eigenvalue problem

TL;DR

This work addresses computing the dominant invariant subspace of a real symmetric matrix by recasting the problem as optimization on the Grassmann manifold Gr of -dimensional subspaces. It develops a gradient descent method and a Polak–Ribiere nonlinear conjugate gradient variant, both with an exact line search, and proves global convergence with local linear convergence in a neighborhood governed by the spectral gap . The authors also provide practical implementation details to ensure accurate line searches and efficient matvecs, and they compare the methods against subspace iteration (with Chebyshev acceleration) and LOBCG across a suite of matrices, showing favorable performance in terms of matrix-vector products and runtime in many cases. These methods are particularly relevant for applications like SCF iterations in electronic structure calculations where spectrum information is not readily available and robustness to matrix changes is valuable.

Abstract

This paper explores variants of the subspace iteration algorithm for computing approximate invariant subspaces. The standard subspace iteration approach is revisited and new variants that exploit gradient-type techniques combined with a Grassmann manifold viewpoint are developed. A gradient method as well as a nonlinear conjugate gradient technique are described. Convergence of the gradient-based algorithm is analyzed and a few numerical experiments are reported, indicating that the proposed algorithms are sometimes superior to standard algorithms. This includes the Chebyshev-based subspace iteration and the locally optimal block conjugate gradient method, when compared in terms of number of matrix vector products and computational time, resp. The new methods, on the other hand, do not require estimating optimal parameters. An important contribution of this paper to achieve this good performance is the accurate and efficient implementation of an exact line search. In addition, new convergence proofs are presented for the non-accelerated gradient method that includes a locally exponential convergence if started in a neighbourhood of the dominant subspace with spectral gap .
Paper Structure (30 sections, 9 theorems, 77 equations, 7 figures, 1 table, 4 algorithms)

This paper contains 30 sections, 9 theorems, 77 equations, 7 figures, 1 table, 4 algorithms.

Table of Contents

  1. Introduction
  2. Background and notation
  3. Invariant subspaces
  4. Notation and assumptions
  5. Subspace iteration
  6. Comparison with Krylov subspace methods
  7. Invariant subspaces and the Grassmannian perspective
  8. Gradient method on Grassmann
  9. Efficient line search
  10. Convergence of the gradient method
  11. Global convergence of the gradient vector field
  12. Local linear convergence
  13. Accelerated gradient method
  14. Polak--Ribiere nonlinear conjugate gradients
  15. Line search
  16. ...and 15 more sections

Key Result

Lemma 1

Define $L \equiv \lambda_{max}(A) - \lambda_{min}(A)$. Then for any given $\mu \ge 0$ the 'loss' term (2nd term in right-hand side of eq:phiconv0) satisfies where $G=\mathop{\mathrm{grad}}\nolimits \phi(X(0))$ and $\beta_{max} = \max \beta_i$.

Figures (7)

  • Figure 1: Illustration of the line search and the tangent space
  • Figure 2: Error in objective value for subspace iteration (SI), Riemannian steepest descent (SD), Riemannian nonlinear conjugate gradients (CG), and locally optimal block conjugate gradients (LOBCG) for a Laplacian matrix of size $n = 1\, 400$ based on finite differences when computing the dominant subspace of dimension $p=6$. For SI, optimal shift and optimal Chebyshev polynomials were used of various degree (number in legend). The black lines estimate the asymptotic convergence speed as explained in the text.
  • Figure 3: The FD3D matrix.
  • Figure 4: The ukerbe1 matrix.
  • Figure 5: The ACTIVSg70K matrix.
  • ...and 2 more figures

Theorems & Definitions (16)

  • Lemma 1
  • proof
  • Lemma 2
  • Lemma 3
  • proof
  • Theorem 4
  • proof
  • Proposition 5
  • proof
  • Proposition 6
  • ...and 6 more