A computational study of low precision incomplete Cholesky factorization preconditioners for sparse linear least-squares problems

TL;DR

The study investigates using low-precision incomplete Cholesky preconditioners to accelerate and stabilize solving sparse linear least-squares problems via LSQR. It compares level-based IC ($IC(\ell)$) and memory-limited IC preconditioners, finding that $IC(\ell)$ is generally ineffective for LS problems while memory-limited IC can provide high-quality preconditioners, particularly when memory is constrained and accuracy requirements are modest. By employing fp16, fp32, and fp64 arithmetic and adaptive stopping criteria such as $ratio_{PT}$, the work shows that half-precision preconditioning can be competitive under tight memory or accuracy constraints, whereas fp32 often achieves comparable convergence with substantial memory savings and robust behavior through breakdown-avoidance strategies. The results guide practical design of mixed-precision LS solvers, highlighting when low-precision preconditioners are viable and how to robustly handle breakdown in incomplete factorizations for ill-conditioned problems.

Abstract

Our interest lies in the robust and efficient solution of large sparse linear least-squares problems. In recent years, hardware developments have led to a surge in interest in exploiting mixed precision arithmetic within numerical linear algebra algorithms to take advantage of potential savings in memory requirements, runtime and energy use, whilst still achieving the requested accuracy. We explore employing mixed precision when solving least-squares problems, focusing on the practicalities of developing robust approaches using low precision incomplete Cholesky factorization preconditioners. Key penalties associated with lower precision include a loss of reliability and less accuracy in the computed solution. Through experiments involving problems from practical applications, we study computing incomplete Cholesky factorizations of the normal matrix using low precision and using the factors to precondition LSQR using mixed precision. We investigate level-based and memory-limited incomplete factorization preconditioners. We find that the former are not effective for least-squares problems while the latter can provide high-quality preconditioners. In particular, half precision arithmetic can be considered if high accuracy is not required in the solution or the memory for the incomplete factors is very restricted; otherwise, single precision can be used, and double precision accuracy recovered while reducing memory consumption, even for ill-conditioned problems.