Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A comparison of mixed precision iterative refinement approaches for least-squares problems

Erin Carson, Ieva Daužickaitė

TL;DR

This work analyzes three mixed-precision iterative refinement strategies for least-squares problems—LS system, semi-normal equations, and augmented system—under two precisions and derives forward-error bounds, recognizability criteria, and convergence conditions. It extends Björck-style uniform-precision results to mixed precision for the semi-normal approach and demonstrates that the augmented-system method is most robust to residual size and ill-conditioning, while the LS approach has limited guarantees. The authors also show how combining LS and augmented schemes enables solving multiple LS problems and refining both the solution and residual with iterative solvers. Numerical experiments illustrate the relative strengths of each method, informing practical selection based on conditioning, residual, and solver availability. Overall, the paper provides a decision framework and new two-precision forward-error analysis to guide efficient, accurate LSIR implementations.

Abstract

Various approaches to iterative refinement (IR) for least-squares problems have been proposed in the literature and it may not be clear which approach is suitable for a given problem. We consider three approaches to IR for least-squares problems when two precisions are used and review their theoretical guarantees, known shortcomings and when the method can be expected to recognize that the correct solution has been found, and extend uniform precision analysis for an IR approach based on the semi-normal equations to the two-precision case. We focus on the situation where it is desired to refine the solution to the working precision level. It is shown that the IR methods exhibit different sensitivities to the conditioning of the problem and the size of the least-squares residual, which should be taken into account when choosing the IR approach. We also discuss a new approach that is based on solving multiple least-squares problems.

A comparison of mixed precision iterative refinement approaches for least-squares problems

TL;DR

This work analyzes three mixed-precision iterative refinement strategies for least-squares problems—LS system, semi-normal equations, and augmented system—under two precisions and derives forward-error bounds, recognizability criteria, and convergence conditions. It extends Björck-style uniform-precision results to mixed precision for the semi-normal approach and demonstrates that the augmented-system method is most robust to residual size and ill-conditioning, while the LS approach has limited guarantees. The authors also show how combining LS and augmented schemes enables solving multiple LS problems and refining both the solution and residual with iterative solvers. Numerical experiments illustrate the relative strengths of each method, informing practical selection based on conditioning, residual, and solver availability. Overall, the paper provides a decision framework and new two-precision forward-error analysis to guide efficient, accurate LSIR implementations.

Abstract

Various approaches to iterative refinement (IR) for least-squares problems have been proposed in the literature and it may not be clear which approach is suitable for a given problem. We consider three approaches to IR for least-squares problems when two precisions are used and review their theoretical guarantees, known shortcomings and when the method can be expected to recognize that the correct solution has been found, and extend uniform precision analysis for an IR approach based on the semi-normal equations to the two-precision case. We focus on the situation where it is desired to refine the solution to the working precision level. It is shown that the IR methods exhibit different sensitivities to the conditioning of the problem and the size of the least-squares residual, which should be taken into account when choosing the IR approach. We also discuss a new approach that is based on solving multiple least-squares problems.
Paper Structure (19 sections, 2 theorems, 83 equations, 14 figures, 1 table, 4 algorithms)

This paper contains 19 sections, 2 theorems, 83 equations, 14 figures, 1 table, 4 algorithms.

Table of Contents

  1. Introduction
  2. Notation
  3. Least-squares system approach
  4. Semi-normal equations approach
  5. Computed $R$-factor of $A$
  6. Correction to the true solution
  7. The forward error
  8. Comparison with corrected semi-normal equations
  9. Augmented system approach
  10. Solving via QR decomposition
  11. Correction to true solution
  12. Initial rate of convergence and sufficient conditions for convergence
  13. Scaling of the augmented system
  14. Iterative solvers
  15. Combining least-squares and augmented system approaches
  16. ...and 4 more sections

Key Result

Theorem 3.1

\newlabelth:semi_normal Let $\widehat{x}_{i}$ be the solution computed in iteration $i$ of Algorithm alg:semi-normal_approach and $x^*$ be the true solution of eq:LS. Then the relative forward error after $i$ iterations is bounded as where $c_1(m,n)$ and $c_2(m,n)$ are constants depending on $m$ and $n$.

Figures (14)

  • Figure 7.1: LS system approach. Criteria for the least-squares system approach to recognize a true solution if it is found \ref{['eq:gol_wilk_condition']} for a set of LS problems with varying $\kappa(A)$ and $\Vert r^* \Vert$. The white cells indicate that \ref{['eq:gol_wilk_condition']} is satisfied.
  • Figure 7.2: Augmented system approach. LSIR convergence criteria \ref{['eq:condition_x_convergence_simplified']} for a set of LS problems with varying $\kappa(A)$ and $\Vert r^* \Vert$ for two sets of precisions $u_r$ and $u$. The white cells indicate that \ref{['eq:condition_x_convergence_simplified']} is satisfied.
  • Figure 7.3: Count of iterative refinement iterations for the LS, semi-normal equations, and augmented system approach when $u_r$ is set to quadruple and $u$ is set to double (left panels), and $u_r$ is set to double and $u$ is set to single (right). Black denotes when LSIR does not converge in 30 iterations.
  • Figure 7.4: As in Figure \ref{['fig:ir_iterations']}, but for the relative error in $x$.
  • Figure 7.5: As in Figure \ref{['fig:ir_iterations']}, but for the relative error in $r$.
  • ...and 9 more figures

Theorems & Definitions (4)

  • Theorem 3.1
  • proof
  • Lemma 4.1
  • proof