Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A mixed precision preconditioned Jacobi method for the symmetric eigenvalue problem

Zhiyuan Zhang, Zheng-Jian Bai

TL;DR

This work analyzes the floating-point behavior of Jacobi rotations and develops mixed-precision preconditioned Jacobi methods for both the symmetric eigenvalue problem and the SVD. The core idea is to compute a low-precision eigen-decomposition to obtain a preliminary eigenvector structure, then enforce high-precision orthogonality via Gram-Schmidt to form an accurate initial guess for Jacobi iterations. The authors provide detailed error bounds for single steps and full sweeps under various orderings, and extend the framework to the SVD with a mixed-precision one-sided Jacobi variant. Comprehensive numerical experiments on CPUs and GPUs demonstrate substantial speedups over classical Jacobi while maintaining high accuracy, highlighting the practical potential of mixed-precision Jacobi methods for large-scale linear algebra tasks.

Abstract

The eigenvalue problem is a fundamental problem in scientific computing. In this paper, we first give the error analysis for a single step or sweep of Jacobi's method in floating point arithmetic. Then we propose a mixed precision preconditioned Jacobi method for the symmetric eigenvalue problem: We first compute the eigenvalue decomposition of a real symmetric matrix by an eigensolver at low precision and we obtain a low-precision matrix of eigenvectors; Then by using the high-precision modified Gram-Schmidt orthogonalization process, a high-precision orthogonal matrix is obtained, which is used as an initial guess for Jacobi's method. The rounding error analysis of the proposed method is established under some conditions. We also present a mixed precision preconditioned one-sided Jacobi method for the singular value problem and the corresponding rounding error analysis is discussed. Numerical experiments on CPUs and GPUs are reported to illustrate the efficiency of the proposed method over the original Jacobi method.

A mixed precision preconditioned Jacobi method for the symmetric eigenvalue problem

TL;DR

This work analyzes the floating-point behavior of Jacobi rotations and develops mixed-precision preconditioned Jacobi methods for both the symmetric eigenvalue problem and the SVD. The core idea is to compute a low-precision eigen-decomposition to obtain a preliminary eigenvector structure, then enforce high-precision orthogonality via Gram-Schmidt to form an accurate initial guess for Jacobi iterations. The authors provide detailed error bounds for single steps and full sweeps under various orderings, and extend the framework to the SVD with a mixed-precision one-sided Jacobi variant. Comprehensive numerical experiments on CPUs and GPUs demonstrate substantial speedups over classical Jacobi while maintaining high accuracy, highlighting the practical potential of mixed-precision Jacobi methods for large-scale linear algebra tasks.

Abstract

The eigenvalue problem is a fundamental problem in scientific computing. In this paper, we first give the error analysis for a single step or sweep of Jacobi's method in floating point arithmetic. Then we propose a mixed precision preconditioned Jacobi method for the symmetric eigenvalue problem: We first compute the eigenvalue decomposition of a real symmetric matrix by an eigensolver at low precision and we obtain a low-precision matrix of eigenvectors; Then by using the high-precision modified Gram-Schmidt orthogonalization process, a high-precision orthogonal matrix is obtained, which is used as an initial guess for Jacobi's method. The rounding error analysis of the proposed method is established under some conditions. We also present a mixed precision preconditioned one-sided Jacobi method for the singular value problem and the corresponding rounding error analysis is discussed. Numerical experiments on CPUs and GPUs are reported to illustrate the efficiency of the proposed method over the original Jacobi method.
Paper Structure (16 sections, 30 theorems, 160 equations, 5 figures, 3 tables, 5 algorithms)

This paper contains 16 sections, 30 theorems, 160 equations, 5 figures, 3 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Jacobi's method in floating point arithmetic
  4. Jacobi's method
  5. Error analysis for one step of Jacobi's method in floating point arithmetic
  6. Error analysis for one sweep of the cyclic Jacobi method in floating point arithmetic
  7. The general cyclic order with distinct eigenvalues
  8. The row-cyclic order with distinct eigenvalues
  9. The row-cyclic order with one multiple eigenvalue
  10. A mixed precision preconditioned Jacobi method for the symmetric eigenvalue problem
  11. A mixed precision preconditioned one-sided Jacobi method for the SVD
  12. Numerical Experiments
  13. The symmetric eigenvalue problem
  14. The singular value problem
  15. Numerical verification
  16. ...and 1 more sections

Key Result

Lemma 2.1

Let $A \in {\mathbb R}^{m\times n}$ with ${\rm rank}(A)=n$. Suppose the MGS method computes the approximate QR factorization $A\approx \hat{Q}\hat{R}$ in precision $\upsilon$, where $\hat{R}\in{\mathbb R}^{n\times n}$ is upper triangular and $\hat{Q}\in{\mathbb R}^{m\times n}$. Then there exist cons

Figures (5)

  • Figure 3.1: The diagram of the row-effected indices (in blue) and the column-effected indices (in red).
  • Figure 6.1: Different quantities versus $n$ for Example \ref{['ex1']}.
  • Figure 6.2: $\| \mathrm{off}(T_{jN}) \|_F$ versus $j$ (sweeps) for Example \ref{['ex2']} with $(r,s)=(2,256)$. Left: $(\tt{mode},\tt{kappa}) =(3,10^4)$ and right: $(\tt{mode}, \tt{kappa}) = (5, 10^4)$.
  • Figure 6.3: $\| \mathrm{off}(T_{jN}) \|_F$ versus $j$ (sweeps) for Example \ref{['ex2']} with $(r,s)=(4,128)$. Left: $(\tt{mode},\tt{kappa}) =(4,10^8)$ and right: $(\tt{mode}, \tt{kappa}) = (4, 10^{12})$.
  • Figure 6.4: $\tilde{\zeta}$ versus $j$ (sweeps) in double precision for Example \ref{['ex1']} with different choices of $(\tt{mode}, \tt{kappa})$. Left: $n=1024$ and right: $n=4096$.

Theorems & Definitions (41)

  • Lemma 2.1
  • Lemma 2.2
  • Lemma 2.3
  • Lemma 2.4
  • Lemma 2.5
  • Lemma 3.1
  • Lemma 3.2
  • Lemma 3.3
  • Lemma 3.4
  • Lemma 3.5
  • ...and 31 more