Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On a shrink-and-expand technique for symmetric block eigensolvers

Yuqi Liu, Yuxin Ma, Meiyue Shao

TL;DR

The paper introduces a shrink-and-expand technique to accelerate symmetric block eigensolvers by adaptively changing the block size during iteration, rather than relying on deflation. It generalizes the idea across SI, SD, LOBPCG, and TraceMIN, proposing three adaptive strategies (fix, slope, slopek) and provides a detailed convergence analysis showing that expansion can restore convergence speed after shrinkage. Theoretical results, complemented by extensive numerical experiments on SPD matrices, demonstrate average speedups of about $20$–$30\%$ (up to $50\%$ in some cases) while maintaining accuracy. The method is memory-efficient and broadly applicable to a range of block eigensolvers, with potential extensions to other Krylov-type methods and non-Hermitian problems.

Abstract

In symmetric block eigenvalue algorithms, such as the subspace iteration algorithm and the locally optimal block preconditioned conjugate gradient (LOBPCG) algorithm, a large block size is often employed to achieve robustness and rapid convergence. However, using a large block size also increases the computational cost. Traditionally, the block size is typically reduced after convergence of some eigenpairs, known as deflation. In this work, we propose a non-deflation-based, more aggressive technique, where the block size is adjusted dynamically during the algorithm. This technique can be applied to a wide range of block eigensolvers, reducing computational cost without compromising convergence speed. We present three adaptive strategies for adjusting the block size, and apply them to four well-known eigensolvers as examples. Detailed theoretical analysis and numerical experiments are provided to illustrate the efficiency of the proposed technique. In practice, an overall acceleration of 20% to 30% is observed.

On a shrink-and-expand technique for symmetric block eigensolvers

TL;DR

The paper introduces a shrink-and-expand technique to accelerate symmetric block eigensolvers by adaptively changing the block size during iteration, rather than relying on deflation. It generalizes the idea across SI, SD, LOBPCG, and TraceMIN, proposing three adaptive strategies (fix, slope, slopek) and provides a detailed convergence analysis showing that expansion can restore convergence speed after shrinkage. Theoretical results, complemented by extensive numerical experiments on SPD matrices, demonstrate average speedups of about – (up to in some cases) while maintaining accuracy. The method is memory-efficient and broadly applicable to a range of block eigensolvers, with potential extensions to other Krylov-type methods and non-Hermitian problems.

Abstract

In symmetric block eigenvalue algorithms, such as the subspace iteration algorithm and the locally optimal block preconditioned conjugate gradient (LOBPCG) algorithm, a large block size is often employed to achieve robustness and rapid convergence. However, using a large block size also increases the computational cost. Traditionally, the block size is typically reduced after convergence of some eigenpairs, known as deflation. In this work, we propose a non-deflation-based, more aggressive technique, where the block size is adjusted dynamically during the algorithm. This technique can be applied to a wide range of block eigensolvers, reducing computational cost without compromising convergence speed. We present three adaptive strategies for adjusting the block size, and apply them to four well-known eigensolvers as examples. Detailed theoretical analysis and numerical experiments are provided to illustrate the efficiency of the proposed technique. In practice, an overall acceleration of 20% to 30% is observed.
Paper Structure (29 sections, 5 theorems, 57 equations, 6 figures, 6 tables, 7 algorithms)

This paper contains 29 sections, 5 theorems, 57 equations, 6 figures, 6 tables, 7 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries on block eigensolvers
  3. Subspace iteration
  4. Steepest descent
  5. LOBPCG
  6. TraceMIN
  7. The shrink-and-expand technique
  8. A general framework
  9. Strategies for applying shrink-and-expand
  10. The fix strategy
  11. The slope strategy
  12. The slopek strategy
  13. An illustrative example for the convergence of the shrink-and-expand technique
  14. Why the convergence rate do not decline immediately after the shrinkage
  15. Why the convergence rate can be restored after the expansion
  16. ...and 14 more sections

Key Result

Proposition 1

Let $A\in\mathbb{C}^{n\times n}$ be a Hermitian positive definite matrix with normalized eigenpairs $(\lambda_1,v_1)$, $(\lambda_2,v_2)$, $\dotsc$, $(\lambda_n,v_n)$. Assume that $0<\lambda_1\leq\lambda_2\leq\dotsb\leq\lambda_n$, and the initial guess $x^{(0)}\in\mathbb{C}^n$ satisfying $v_1^* x^{(0 where $x^{(j)}$ is the approximate eigenvector of iteration $j$, and $\theta^{(j)}$ represents the

Figures (6)

  • Figure 1: Left: Use the SI algorithm to approximate five largest eigenpairs of the matrix bcsstk08. Here, ${n_{\mathrm{ev}}}=5$ is the number of desired eigenpairs, and ${n_{\mathrm{ex}}}$ is the block size. The number of iterations to converge decreases rapidly when ${n_{\mathrm{ex}}}$ becomes larger. Right: Use a block size of ${n_{\mathrm{ex}}}=4\cdot{n_{\mathrm{ev}}}$ for the first five iterations, and then reduce the block size down to ${n_{\mathrm{ex}}}={n_{\mathrm{ev}}}$ by selecting eigenvectors corresponding to the ${n_{\mathrm{ex}}}$ largest eigenvalues. The high convergence rate is maintained for a few more iterations.
  • Figure 2: Use the SI algorithm to approximate five largest eigenpairs of the matrix bcsstk08, with ${n_{\mathrm{ex}}}=4\cdot{n_{\mathrm{ev}}}$. Different from the curve with ${n_{\mathrm{ex}}}=4\cdot{n_{\mathrm{ev}}}$ in Figure \ref{['fig:intro1']}, the block size is increased at the $12$th iteration and, again, is decreased at the $14$th iteration. The convergence rate recovers rapidly after the expansion.
  • Figure 3: The process of employing the shrink-and-expand technique in block eigensolvers. The size of $X$ decreases from $n\times{n_{\mathrm{ex}}}$ to $n\times{n_{\mathrm{es}}}$ after shrinkage, allowing the iterations marked with the thick arrow to benefit from the reduced cost. To prevent the convergence rate from deteriorating, a larger block size is restored periodically.
  • Figure 4: Using the LOBPCG algorithm to compute $100$ smallest eigenpairs of the (shifted) Muu matrix. Three different shrinkage and expansion strategies---fix, slope, and slopek are applied, respectively.
  • Figure 5: Using the SD algorithm to compute $100$ smallest eigenpairs of the (shifted) Muu matrix. Three different shrink-and-expand strategies---fix, slope, and slopek are applied, respectively. Both the shrinkage and the expansion are very dense in the slope strategy, making it hard to distinguish the curve. Therefore, we have magnified the curve between iterations $645$--$745$ in the lower part to better illustrate the convergence history.
  • ...and 1 more figures

Theorems & Definitions (13)

  • Proposition 1
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Theorem 4
  • Theorem 5
  • proof
  • Remark 1
  • Remark 2
  • ...and 3 more