Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Developing robust incomplete Cholesky factorizations in half precision arithmetic

Jennifer Scott, Miroslav Tůma

TL;DR

This work investigates the robustness of incomplete Cholesky factorizations for ill-conditioned symmetric positive definite matrices when computed in half precision and refined to double precision via mixed-precision iterations. It examines breakdown mechanisms (B1–B3) and introduces strategies—prescaling, look-ahead, global shifting, and local modifications—to prevent or manage breakdowns, plus a five-precision Krylov refinement path to recover high-accuracy solutions. Numerical experiments demonstrate that prescaling and look-ahead improve robustness in low precision, while growth in factor entries remains a risk and must be controlled; the five-precision approach shows promise for achieving double-precision accuracy with low-precision preconditioners. Overall, the results indicate that robust half-precision IC preconditioners are feasible for large sparse SPD systems, with practical guidance on when and how to apply these strategies and valuable directions for future hardware-aware precision hierarchies.

Abstract

Incomplete factorizations have long been popular general-purpose algebraic preconditioners for solving large sparse linear systems of equations. Guaranteeing the factorization is breakdown free while computing a high quality preconditioner is challenging. A resurgence of interest in using low precision arithmetic makes the search for robustness more important and more challenging. In this paper, we focus on ill-conditioned symmetric positive definite problems and explore a number of approaches for preventing and handling breakdowns: prescaling of the system matrix, a look-ahead strategy to anticipate breakdown as early as possible, the use of global shifts, and a modification of an idea developed in the field of numerical optimization for the complete Cholesky factorization of dense matrices. Our numerical simulations target highly ill-conditioned sparse linear systems with the goal of computing the factors in half precision arithmetic and then achieving double precision accuracy using mixed precision refinement. We also consider the often overlooked issue of growth in the sizes of entries in the factors that can occur when using any precision and can render the computed factors ineffective as preconditioners.

Developing robust incomplete Cholesky factorizations in half precision arithmetic

TL;DR

This work investigates the robustness of incomplete Cholesky factorizations for ill-conditioned symmetric positive definite matrices when computed in half precision and refined to double precision via mixed-precision iterations. It examines breakdown mechanisms (B1–B3) and introduces strategies—prescaling, look-ahead, global shifting, and local modifications—to prevent or manage breakdowns, plus a five-precision Krylov refinement path to recover high-accuracy solutions. Numerical experiments demonstrate that prescaling and look-ahead improve robustness in low precision, while growth in factor entries remains a risk and must be controlled; the five-precision approach shows promise for achieving double-precision accuracy with low-precision preconditioners. Overall, the results indicate that robust half-precision IC preconditioners are feasible for large sparse SPD systems, with practical guidance on when and how to apply these strategies and valuable directions for future hardware-aware precision hierarchies.

Abstract

Incomplete factorizations have long been popular general-purpose algebraic preconditioners for solving large sparse linear systems of equations. Guaranteeing the factorization is breakdown free while computing a high quality preconditioner is challenging. A resurgence of interest in using low precision arithmetic makes the search for robustness more important and more challenging. In this paper, we focus on ill-conditioned symmetric positive definite problems and explore a number of approaches for preventing and handling breakdowns: prescaling of the system matrix, a look-ahead strategy to anticipate breakdown as early as possible, the use of global shifts, and a modification of an idea developed in the field of numerical optimization for the complete Cholesky factorization of dense matrices. Our numerical simulations target highly ill-conditioned sparse linear systems with the goal of computing the factors in half precision arithmetic and then achieving double precision accuracy using mixed precision refinement. We also consider the often overlooked issue of growth in the sizes of entries in the factors that can occur when using any precision and can render the computed factors ineffective as preconditioners.
Paper Structure (10 sections, 1 theorem, 16 equations, 6 tables, 5 algorithms)

This paper contains 10 sections, 1 theorem, 16 equations, 6 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Possible breakdowns within IC factorizations
  3. Preventing and handling breakdown in IC factorizations
  4. Avoiding breakdown by prescaling
  5. Preventing breakdown by incorporating look-ahead
  6. Global shifting to handle breakdown
  7. Local modifications to prevent breakdown
  8. Recovering double precision accuracy
  9. Numerical experiments
  10. Concluding remarks

Key Result

Lemma 1

Let the matrix $A$ be sparse and SPD. Assume that, using the GMW($\beta$) strategy, columns 1 to $j-1$ columns of the IC factor $L$ have been successfully computed in fp16 arithmetic. For $i \ge j$ let $nz(i)$ denote the number of nonzero entries in $L_{i,1:j-1}$. If where $x_{max}$ is the largest finite number represented in fp16, then B3 breakdown cannot occur in the $j$-th step.

Theorems & Definitions (2)

  • Lemma 1
  • proof