Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Computing accurate singular values using a mixed-precision one-sided Jacobi algorithm

Zhengbo Zhou, Françoise Tisseur, Marcus Webb

TL;DR

A relative forward error analysis of a mixed-precision preconditioned one-sided Jacobi algorithm, analogous to a two-sided version introduced in [N. J. Higham, F. Tisseur, M. Webb and Z. Zhou], which uses low precision to compute the preconditioner, applies it in high precision, and computes the singular value decomposition using the one-sided Jacobi algorithm at working precision is presented.

Abstract

We present a relative forward error analysis of a mixed-precision preconditioned one-sided Jacobi algorithm, analogous to a two-sided version introduced in [N. J. Higham, F. Tisseur, M. Webb and Z. Zhou, SIAM J. Matrix Anal. Appl. 46 (2025), pp. 2423-2448], which uses low precision to compute the preconditioner, applies it in high precision, and computes the singular value decomposition using the one-sided Jacobi algorithm at working precision. Our analysis yields smaller relative forward error bounds for the computed singular values than those of standard SVD algorithms. We present and analyse two approaches for constructing effective preconditioners. Our numerical experiments support the theoretical results and demonstrate that our algorithm achieves smaller relative forward errors than the LAPACK routines $\texttt{DGESVJ}$ and $\texttt{DGEJSV}$, as well as the MATLAB function $\texttt{svd}$, particularly for ill-conditioned matrices. Timing tests show that our approach accelerates the convergence of the Jacobi iterations and that the dominant cost arises from a single high-precision matrix-matrix multiplication. With improved software or hardware support for this bottleneck, our algorithm would be faster than the LAPACK one-sided Jacobi algorithm $\texttt{DGESVJ}$ and comparable in speed to the state-of-the-art preconditioned one-sided Jacobi algorithm $\texttt{DGEJSV}$, but much more accurate.

Computing accurate singular values using a mixed-precision one-sided Jacobi algorithm

TL;DR

A relative forward error analysis of a mixed-precision preconditioned one-sided Jacobi algorithm, analogous to a two-sided version introduced in [N. J. Higham, F. Tisseur, M. Webb and Z. Zhou], which uses low precision to compute the preconditioner, applies it in high precision, and computes the singular value decomposition using the one-sided Jacobi algorithm at working precision is presented.

Abstract

We present a relative forward error analysis of a mixed-precision preconditioned one-sided Jacobi algorithm, analogous to a two-sided version introduced in [N. J. Higham, F. Tisseur, M. Webb and Z. Zhou, SIAM J. Matrix Anal. Appl. 46 (2025), pp. 2423-2448], which uses low precision to compute the preconditioner, applies it in high precision, and computes the singular value decomposition using the one-sided Jacobi algorithm at working precision. Our analysis yields smaller relative forward error bounds for the computed singular values than those of standard SVD algorithms. We present and analyse two approaches for constructing effective preconditioners. Our numerical experiments support the theoretical results and demonstrate that our algorithm achieves smaller relative forward errors than the LAPACK routines and , as well as the MATLAB function , particularly for ill-conditioned matrices. Timing tests show that our approach accelerates the convergence of the Jacobi iterations and that the dominant cost arises from a single high-precision matrix-matrix multiplication. With improved software or hardware support for this bottleneck, our algorithm would be faster than the LAPACK one-sided Jacobi algorithm and comparable in speed to the state-of-the-art preconditioned one-sided Jacobi algorithm , but much more accurate.
Paper Structure (22 sections, 13 theorems, 84 equations, 5 figures, 3 algorithms)

This paper contains 22 sections, 13 theorems, 84 equations, 5 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Mixed-precision preconditioned one-sided Jacobi algorithm
  3. The main theorem
  4. Analysis of E1
  5. Bound on the forward error E2
  6. Intermediate bound
  7. The size of alpha and choice of uh
  8. Relationship between $\kappa_{2}^{D}({\widetilde{A}})$ and $\kappa_{2}^{D}({\widetilde{A}_{\mathrm{comp}}})$
  9. Construction of the preconditioner
  10. Bounding $\kappa_{2}^{D}({\widetilde{A}})$
  11. Bounding $\operatorname{obliq}({\widetilde{A}})$
  12. Using an initial QR factorization step
  13. For the one-sided Jacobi algorithm
  14. QR before preconditioning
  15. QR after preconditioning
  16. ...and 7 more sections

Key Result

Theorem 2.3

Let $A \in \mathbb{R}^{m\times n}$$(m\ge n)$ be of full rank, and let ${\widetilde{A}} = A\widetilde{V}$ be the preconditioned matrix as in Algorithm alg.mp-precond-onesided-Jacobi. If Assumption ass.main holds, then the forward error ${{\varepsilon}_{fwd}^{(k)}}$ defined in eq.ferrk satisfies provided that both terms on the right-hand side are less than $1$.

Figures (5)

  • Figure 1: Maximum relative forward error $\max_k {{\varepsilon}_{fwd}^{(k)}}$ for applying MP3JacobiSVD, , and MATLAB svd on matrices $A \in \mathbb{R}^{1000\times800}$, $\kappa_2(A)\in[10^3, 10^{15}]$ and MODE as indicated in the subcaptions.
  • Figure 2: Maximum relative forward error $\max_k {{\varepsilon}_{fwd}^{(k)}}$ for applying MP3JacobiSVD, , and MATLAB svd on matrices with a fixed number of rows $m = 10^3$, a varying number of columns $n$ from $10$ to $10^3$, a fixed condition number $10^{8}$ and MODE as indicated by the subcaptions.
  • Figure 3: Relative forward error ${{\varepsilon}_{fwd}^{(k)}}$ for applying MP3JacobiSVD, , and MATLAB svd on four test matrices from Anymatrix toolbox as described in section \ref{['sec.numerical-experiment.special-test']} and indicated by the subcaptions. Figure \ref{['subfig.gallery/lauchli']} only displays ${{\varepsilon}_{fwd}^{(k)}}$ for every tenth singular value, and the missing data points are because their ${{\varepsilon}_{fwd}^{(k)}}$ values were below $10^{-16}$.
  • Figure 4: Timing tests for MP3JacobiSVD, , , MATLAB svd, and a theoretical variant of MP3JacobiSVD in which the preconditioner applied in quadruple precision is only $100$ times slower than in double precision. Figure \ref{['subfig.relative-timing-for-preconditioning']} (top-right) shows the time elapsed applying the preconditioner at quadruple precision divided by that at double precision. The test matrices have a fixed number of rows $m = 3000$, a varying number of columns $n$ from $100$ to $3000$, a fixed condition number $10^8$, and geometrically distributed singular values.
  • Figure 5: Relative forward accuracy and runtime tests for MPJacobiSVD_SSD (MP3JacobiSVD with single, single and double precision as low, working and high precision, respectively), and applied on test matrices with a fixed number of rows $m=3000$, a varying number of columns $n$ from $100$ to $3000$, a fixed condition number of $10^6$, and geometrically distributed singular values.

Theorems & Definitions (29)

  • Remark 2.2
  • Theorem 2.3
  • Lemma 2.4
  • Proof 1
  • Lemma 2.5
  • Proof 2
  • Lemma 2.6
  • Proof 3
  • Theorem 2.7: deve92
  • Lemma 2.8
  • ...and 19 more