Randomized Householder QR

TL;DR

This work introduces Randomized Householder QR (RHQR), a factorization that replaces the traditional QR of a tall-and-skinny matrix with a sketched counterpart, yielding a well-conditioned basis whose sketch is near-orthogonal and whose factorization inherits Householder QR stability. By linking RHQR to the QR of the sketched problem via a sketching matrix $\Psi$, the authors derive left-looking and right-looking RHQR algorithms, a reconstruction method (recRHQR), and a BLAS2-RGS variant, all supported by finite-precision analysis under mild probabilistic assumptions. The approach enables efficient, synchronized computation and can be integrated into Krylov subspace methods (Arnoldi and GMRES), with stability and accuracy demonstrated through SRHT-based sketches and extensive numerical experiments, including mixed and half-precision settings. The proposed methods promise improved stability and reduced communication in large-scale linear-algebra tasks, potentially benefiting randomized QR, QR with column pivoting, and QR-based Krylov solvers in high-performance computing contexts.

Abstract

This paper introduces a randomized Householder QR factorization (RHQR). This factorization can be used to obtain a well conditioned basis of a vector space and thus can be employed in a variety of applications. The RHQR factorization of the input matrix $W$ is equivalent to the standard Householder QR factorization of matrix $ΨW$, where $Ψ$ is a sketching matrix that can be obtained from any subspace embedding technique. For this reason, the RHQR factorization can also be reconstructed from the Householder QR factorization of the sketched problem, yielding a single-synchronization randomized QR factorization (recRHQR). In most contexts, left-looking RHQR requires a single synchronization per iteration, with half the computational cost of Householder QR, and a similar cost to Randomized Gram-Schmidt (RGS) overall. We discuss the usage of RHQR factorization in the Arnoldi process and then in GMRES, showing thus how it can be used in Krylov subspace methods to solve systems of linear equations. Based on Charles Sheffield's connection between Householder QR and Modified Gram-Schmidt (MGS), a BLAS2-RGS is also derived. A finite precision analysis shows that, under mild probabilistic assumptions, the RHQR factorization of the input matrix $W$ inherits the stability of the Householder QR factorization, producing a well-conditioned basis and a columnwise backward stable factorization, all independently of the condition number of the input $W$, and with the accuracy of the sketching step. We study the subsampled randomized Hadamard transform (SRHT) as a very stable sketching technique. Numerical experiments show that RHQR produces a well conditioned basis whose sketch is numerically orthogonal and an accurate factorization, even for the most difficult inputs and with high-dimensional operations made in half-precision.

Figures (9)

• Figure 1: Unique precision $\mathbf{e}$
• Figure 2: Randomized Householder QR in double precision (RHVector in the graphs) and comparison with RGS.
• Figure 3: Randomized Householder QR in single precision (RHouse in the graphs) and comparison with RGS.
• Figure 4: Randomized Householder QR compared to RGS in mixed half/double precision.
• Figure 5: Performance of recRHQR compared to Randomized Cholesky QR

Theorems & Definitions (40)

• Definition 2.1
• Definition 2.2
• Proposition 3.1
• proof
• Remark 3.2
• Remark 3.3
• Proposition 3.4
• proof
• Remark 3.5
• Proposition 3.6