Mixed Precision FGMRES-Based Iterative Refinement for Weighted Least Squares

TL;DR

The paper addresses solving weighted least squares problems $\min_x \|D^{1/2}(b-Ax)\|_2$ efficiently with mixed-precision hardware by developing a FGMRES-based iterative refinement approach (FGMRES-WLSIR) that extends GMRES-LSIR to the weighted setting. It introduces two preconditioners—a left QR-based preconditioner $M_l$ and a block-diagonal (split) preconditioner $M_b$—and provides analytical bounds on the conditioning of the preconditioned augmented system, highlighting how dependence on the weight matrix $D$ affects stability. Numerical experiments on synthetic and SuiteSparse matrices illustrate the trade-offs: the left preconditioner can be fragile to ill-conditioned $D$ in low precision, while the split preconditioner can mitigate $D$-dependence but introduces sensitivity to the right preconditioner, making performance problem-dependent. The study concludes that no single preconditioner universally dominates; careful selection or alternative approaches may be needed depending on $D$ and problem conditioning, with potential for using half-precision in favorable cases. Overall, the work advances robust mixed-precision techniques for weighted least squares through FGMRES-WLSIR and informs practical preconditioner choices for modern hardware.

Abstract

With the recent emergence of mixed precision hardware, there has been a renewed interest in its use for solving numerical linear algebra problems fast and accurately. The solution of least squares (LS) problems $\min_x\|b-Ax\|_2$, where $A \in \mathbb{R}^{m\times n}$, arise in numerous application areas. Overdetermined standard least squares problems can be solved by using mixed precision within the iterative refinement method of Björck, which transforms the least squares problem into an $(m+n)\times(m+n)$ ''augmented'' system. It has recently been shown that mixed precision GMRES-based iterative refinement can also be used, in an approach termed GMRES-LSIR. In practice, we often encounter types of least squares problems beyond standard least squares, including weighted least squares (WLS), $\min_x\|D^{1/2}(b-Ax)\|_2$, where $D^{1/2}$ is a diagonal matrix of weights. In this paper, we discuss a mixed precision FGMRES-WLSIR algorithm for solving WLS problems using two different preconditioners.

Figures (3)

• Figure 1: Measured condition number of the preconditioned systems and estimates of the bound on the condition number \ref{['eq:kinfbound']} where the preconditioners \ref{['eq:M']} are constructing using QR factorizations computed in various precisions, versus the condition number of the weight matrix $D$. The working precision for FGMRES-WLSIR is assumed to be single precision.
• Figure 2: Measured condition number of the preconditioners (left) and preconditioned systems (right) using ash958 matrix as $A$ where $M_l$ and $M_b$ are constructed using QR factorizations in various precisions, versus the condition number of the weight matrix $D$. The working precision for FGMRES-WLSIR is assumed to be double precision.
• Figure 3: Measured condition number of the preconditioners (left) and preconditioned systems (right) using robot24c1_mat5 matrix as $A$ where $M_l$ and $M_b$ are constructed using QR factorizations in various precisions, versus the condition number of the weight matrix $D$. The working precision for FGMRES-WLSIR is assumed to be double precision.