Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EPIC: a provable accelerated Eigensolver based on Preconditioning and Implicit Convexity

Nian Shao, Wenbin Chen, Zhaojun Bai

TL;DR

This paper addresses the problem of efficiently computing the smallest eigenpair of a symmetric positive definite pencil $(A,M)$ by revealing an implicit convexity structure in Euclidean space. It develops EPIC, a preconditioned Eigensolver based on Implicit Convexity, and couples it with a Locally Optimal Nesterov Accelerated Gradient (LONAG) framework to achieve accelerated convergence similar to the (unproven) expected rate of LO preconditioned CG. The key contributions are: (i) constructing an auxiliary convex problem on a tangent-like space linked to the Rayleigh quotient, (ii) proving convexity and convergence properties of the auxiliary problem via the maps $\psi$ and $\psi^{\dagger}$, (iii) formulating EIC and EPIC with rigorous convergence bounds, and (iv) providing numerical experiments on test matrices to validate acceleration and compare with LOPCG. The approach yields practical impact by offering a theoretically grounded, accelerated solver for large-scale eigenvalue problems where preconditioning plays a central role, and it suggests promising directions for parameter-free and block-iterative extensions.

Abstract

This paper is concerned with the extraction of the smallest eigenvalue and the corresponding eigenvector of a symmetric positive definite matrix pencil. We reveal implicit convexity of the eigenvalue problem in Euclidean space. A provable accelerated eigensolver based on preconditioning and implicit convexity (EPIC) is proposed. Theoretical analysis shows the acceleration of EPIC with the rate of convergence resembling the expected rate of convergence of the well-known locally optimal preconditioned conjugate gradient (LOPCG). A complete proof of the expected rate of convergence of LOPCG is elusive so far. Numerical results confirm our theoretical findings of EPIC.

EPIC: a provable accelerated Eigensolver based on Preconditioning and Implicit Convexity

TL;DR

This paper addresses the problem of efficiently computing the smallest eigenpair of a symmetric positive definite pencil by revealing an implicit convexity structure in Euclidean space. It develops EPIC, a preconditioned Eigensolver based on Implicit Convexity, and couples it with a Locally Optimal Nesterov Accelerated Gradient (LONAG) framework to achieve accelerated convergence similar to the (unproven) expected rate of LO preconditioned CG. The key contributions are: (i) constructing an auxiliary convex problem on a tangent-like space linked to the Rayleigh quotient, (ii) proving convexity and convergence properties of the auxiliary problem via the maps and , (iii) formulating EIC and EPIC with rigorous convergence bounds, and (iv) providing numerical experiments on test matrices to validate acceleration and compare with LOPCG. The approach yields practical impact by offering a theoretically grounded, accelerated solver for large-scale eigenvalue problems where preconditioning plays a central role, and it suggests promising directions for parameter-free and block-iterative extensions.

Abstract

This paper is concerned with the extraction of the smallest eigenvalue and the corresponding eigenvector of a symmetric positive definite matrix pencil. We reveal implicit convexity of the eigenvalue problem in Euclidean space. A provable accelerated eigensolver based on preconditioning and implicit convexity (EPIC) is proposed. Theoretical analysis shows the acceleration of EPIC with the rate of convergence resembling the expected rate of convergence of the well-known locally optimal preconditioned conjugate gradient (LOPCG). A complete proof of the expected rate of convergence of LOPCG is elusive so far. Numerical results confirm our theoretical findings of EPIC.
Paper Structure (37 sections, 19 theorems, 160 equations, 2 figures, 3 tables)

This paper contains 37 sections, 19 theorems, 160 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Characterizations and condition number of strongly convex functions.
  4. Paper organization.
  5. Notation.
  6. Implicit convexity of the symmetric eigenvalue problem
  7. The eigenvalue problem
  8. The auxiliary problem
  9. Properties of $\psi$ and $\psi^{\dagger}$
  10. Convexity of $\mathcal{Y}$
  11. The convexity of $\phi$ on $\mathcal{Y}$
  12. Implicit convexity of the eigenvalue problem
  13. Locally Optimal Nestrov Accelerated Gradient descent methods for convex optimization
  14. NAG methods with a dynamical system analogy
  15. Dynamical system.
  16. ...and 22 more sections

Key Result

Proposition 1.1

Suppose $\phi$ is a smooth function on a convex domain $\mathcal{Y}$, and $0<\mu\leq L$ are positive scalars, the following three inequalities for characterizing the strongly convexity of $\phi$ are equivalent: where $y, y_{1}$, $y_{2}\in \mathcal{Y}$.

Figures (2)

  • Figure 1: The relationship of $\mathcal{S}_{+}^{n-1}$, $\mathcal{X}$, $\mathcal{Y}$, $q$, $u_{1}$ and $\psi(u_{1})$
  • Figure 2: Convergence history of LOPCG (red) and EPIC (blue). The $x$-axis is the iterations number. The solid lines are the relative errors of approximate smallest eigenvalues, and the dashed lines are $1-\lvert x_{k}^{\mathsf{T}}Mu_{1}\rvert$, where $x_{k}$ and $u_{1}$ are both $M$--normalized. The restart points are marked by circle.

Theorems & Definitions (38)

  • Proposition 1.1
  • Lemma 2.1
  • proof
  • Proposition 2.1
  • proof
  • Theorem 2.1
  • proof
  • Theorem 2.2
  • Remark 2.1
  • Lemma 2.2
  • ...and 28 more