Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Double precision is not necessary for LSQR for solving discrete linear ill-posed problems

Haibo Li

TL;DR

This paper addresses solving discrete linear ill-posed problems with LSQR under mixed precision by analyzing two main components: the Lanczos-based Krylov subspace construction and the iterative update of the regularized solution. It provides theoretical bounds showing that, for not extremely small noise, the Lanczos vector construction can be performed in single precision without compromising the final accuracy, and that the updating step can also use single precision under a moderate condition number, leading to a practical mixed-precision LSQR with double-precision QR. The authors validate these findings through 1-D and 2-D numerical experiments, demonstrating that mixed-precision variants can achieve the same regularized solutions as the double-precision baseline, thereby offering substantial speedups in large-scale problems. The work delivers actionable guidelines for choosing precision in LSQR and points to future high-performance implementations on architectures supporting low-precision arithmetic.

Abstract

The growing availability and usage of low precision foating point formats has attracts many interests of developing lower or mixed precision algorithms for scientific computing problems. In this paper we investigate the possibility of exploiting lower precision computing in LSQR for solving discrete linear ill-posed problems. We analyze the choice of proper computing precisions in the two main parts of LSQR, including the construction of Lanczos vectors and updating procedure of iterative solutions. We show that, under some mild conditions, the Lanczos vectors can be computed using single precision without loss of any accuracy of final regularized solutions as long as the noise level is not extremely small. We also show that the most time consuming part for updating iterative solutions can be performed using single precision without sacrificing any accuracy. The results indicate that the most time consuming parts of the algorithm can be implemented using single precision, and thus the performance of LSQR for solving discrete linear ill-posed problems can be significantly enhanced. Numerical experiments are made for testing the single precision variants of LSQR and confirming our results.

Double precision is not necessary for LSQR for solving discrete linear ill-posed problems

TL;DR

This paper addresses solving discrete linear ill-posed problems with LSQR under mixed precision by analyzing two main components: the Lanczos-based Krylov subspace construction and the iterative update of the regularized solution. It provides theoretical bounds showing that, for not extremely small noise, the Lanczos vector construction can be performed in single precision without compromising the final accuracy, and that the updating step can also use single precision under a moderate condition number, leading to a practical mixed-precision LSQR with double-precision QR. The authors validate these findings through 1-D and 2-D numerical experiments, demonstrating that mixed-precision variants can achieve the same regularized solutions as the double-precision baseline, thereby offering substantial speedups in large-scale problems. The work delivers actionable guidelines for choosing precision in LSQR and points to future high-performance implementations on architectures supporting low-precision arithmetic.

Abstract

The growing availability and usage of low precision foating point formats has attracts many interests of developing lower or mixed precision algorithms for scientific computing problems. In this paper we investigate the possibility of exploiting lower precision computing in LSQR for solving discrete linear ill-posed problems. We analyze the choice of proper computing precisions in the two main parts of LSQR, including the construction of Lanczos vectors and updating procedure of iterative solutions. We show that, under some mild conditions, the Lanczos vectors can be computed using single precision without loss of any accuracy of final regularized solutions as long as the noise level is not extremely small. We also show that the most time consuming part for updating iterative solutions can be performed using single precision without sacrificing any accuracy. The results indicate that the most time consuming parts of the algorithm can be implemented using single precision, and thus the performance of LSQR for solving discrete linear ill-posed problems can be significantly enhanced. Numerical experiments are made for testing the single precision variants of LSQR and confirming our results.
Paper Structure (10 sections, 7 theorems, 81 equations, 10 figures, 4 tables, 3 algorithms)

This paper contains 10 sections, 7 theorems, 81 equations, 10 figures, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Regularization of linear ill-posed problems and LSQR
  4. Finite precision computing
  5. Choice of computing precision for the construction of Krylov subspace
  6. Updating $x_k$ using lower precision
  7. Numerical Experiments
  8. One dimensional case
  9. Two dimensional case
  10. Conclusion

Key Result

theorem 2.1

For the $k$-step Lanczos bidiagonalization with reorthogonalization, if $\nu_{k+1}<1/2$ and $\mu_{k+1} < 1/2$, then there exist two orthornormal matrices $\bar{P}_{k+1}=(\bar{p}_{1},\dots,\bar{p}_{k+1})\in\mathbb{R}^{m\times(k+1)}$ and $\bar{Q}_{k+1}=(\bar{q}_{1},\dots,\bar{q}_{k+1})\in \mathbb{R}^{ where $E$ and $\delta_b$ are perturbation matrix and vector, respectively. We have error bounds an

Figures (10)

  • Figure 1: Semi-convergence curves for LSQR implemented using different computing precisions, $\varepsilon=10^{-3}$.
  • Figure 2: Relative errors of the regularized solutions computed by "s+d"/"s+s" with respect to that by "d", $\varepsilon=10^{-3}$.
  • Figure 3: Relative errors between regularized solutions computed by "s+d" and "s+s", $\varepsilon=10^{-3}$.
  • Figure 4: Semi-convergence curves for LSQR implemented using different computing precisions, $\varepsilon=10^{-5}$.
  • Figure 5: Semi-convergence curves for LSQR implemented using different computing precisions, $\varepsilon=10^{-7}$.
  • ...and 5 more figures

Theorems & Definitions (13)

  • theorem 2.1
  • theorem 3.1
  • proof
  • lemma thmcounterlemma
  • proof
  • lemma thmcounterlemma
  • proof
  • remark thmcounterremark
  • lemma thmcounterlemma
  • proof
  • ...and 3 more