Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Efficient Batch Solver for the Singular Value Decomposition on GPUs

Ahmad Abdelfattah, Massimiliano Fasi

TL;DR

This work presents a high-performance batch SVD solver for GPUs based on a one-sided Jacobi algorithm. By blocking, parallel round-robin ordering, and a suite of optimizations— including optimized Gram-matrix computation, a dedicated vector-update kernel, and masked batch execution—the solver achieves robust accuracy across real and complex data in multiple precisions and problem shapes. It demonstrates substantial speedups over vendor libraries and prior open-source solvers on both NVIDIA and AMD GPUs, while maintaining numerical reliability. The results underscore the viability of Jacobi-type methods for batched, small-to-medium SVDs on accelerators and highlight transferable design principles for batch linear algebra on GPUs.

Abstract

The singular value decomposition (SVD) is a powerful tool in modern numerical linear algebra, which underpins computational methods such as principal component analysis (PCA), low-rank approximations, and randomized algorithms. Many practical scenarios require solving numerous small SVD problems, a regime generally referred to as "batch SVD". Existing programming models can handle this efficiently on parallel CPU architectures, but high-performance solutions for GPUs remain immature. A GPU-oriented batch SVD solver is introduced. This solver exploits the one-sided Jacobi algorithm to exploit fine-grained parallelism, and a number of algorithmic and design optimizations achieve unmatched performance. Starting from a baseline solver, a sequence of optimizations is applied to obtain incremental performance gains. Numerical experiments show that the new solver is robust across problems with different numerical properties, matrix shapes, and arithmetic precisions. Performance benchmarks on both NVIDIA and AMD systems show significant performance speedups over vendor solutions as well as existing open-source solvers.

An Efficient Batch Solver for the Singular Value Decomposition on GPUs

TL;DR

This work presents a high-performance batch SVD solver for GPUs based on a one-sided Jacobi algorithm. By blocking, parallel round-robin ordering, and a suite of optimizations— including optimized Gram-matrix computation, a dedicated vector-update kernel, and masked batch execution—the solver achieves robust accuracy across real and complex data in multiple precisions and problem shapes. It demonstrates substantial speedups over vendor libraries and prior open-source solvers on both NVIDIA and AMD GPUs, while maintaining numerical reliability. The results underscore the viability of Jacobi-type methods for batched, small-to-medium SVDs on accelerators and highlight transferable design principles for batch linear algebra on GPUs.

Abstract

The singular value decomposition (SVD) is a powerful tool in modern numerical linear algebra, which underpins computational methods such as principal component analysis (PCA), low-rank approximations, and randomized algorithms. Many practical scenarios require solving numerous small SVD problems, a regime generally referred to as "batch SVD". Existing programming models can handle this efficiently on parallel CPU architectures, but high-performance solutions for GPUs remain immature. A GPU-oriented batch SVD solver is introduced. This solver exploits the one-sided Jacobi algorithm to exploit fine-grained parallelism, and a number of algorithmic and design optimizations achieve unmatched performance. Starting from a baseline solver, a sequence of optimizations is applied to obtain incremental performance gains. Numerical experiments show that the new solver is robust across problems with different numerical properties, matrix shapes, and arithmetic precisions. Performance benchmarks on both NVIDIA and AMD systems show significant performance speedups over vendor solutions as well as existing open-source solvers.
Paper Structure (29 sections, 22 equations, 17 figures, 1 table, 3 algorithms)

This paper contains 29 sections, 22 equations, 17 figures, 1 table, 3 algorithms.

Table of Contents

  1. Computing the Singular Value Decomposition
  2. Related Work
  3. Algorithmic Background
  4. Jacobi Hermitian eigensolver
  5. One-sided Jacobi SVD algorithms
  6. Efficient implementations
  7. Experimental Setup
  8. Initial Design
  9. Parallel Ordering
  10. Computation of the Gram Matrices
  11. Hermitian Eigenvalue Problem
  12. Vector Updates
  13. Performance of the Baseline Implementation
  14. Inexact Hermitian Eigendecomposition
  15. Motivation
  16. ...and 14 more sections

Figures (17)

  • Figure 1: Example of the parallel ordering with eight block columns. Each Jacobi sweep consists of seven iterations. Each iteration contains four disjoint pairs of block-columns.
  • Figure 2: Computing the Gram matrix using three concurrent matrix multiplications
  • Figure 3: Heatmaps showing the speedups for MAGMA's own batch GEMM kernel over the vendor's BLAS library for the use case of computing relatively small Gram matrices. Results are shown for double precision on the GH200 system (left) and the MI300A APU (right). Green color (speedups $>1$) means that using MAGMA is recommended for the given dimensions. Red color (speedups $<1$) means that the vendor BLAS is used.
  • Figure 4: Time-to-solution of MAGMA's batch Hermitian eigensolver, with vectors computed. Results are shown for $1000$ double-precision matrices, on a GH200 system (left) and on an MI300A APU (right).
  • Figure 5: GEMM shape for updating the singular vectors.
  • ...and 12 more figures