Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Three-precision iterative refinement with parameter regularization and prediction for solving large sparse linear systems

Jifeng Ge, Juan Zhang

TL;DR

GADI-IR introduces a three-precision iterative refinement strategy within the GADI framework to solve large sparse linear systems Ax = b by performing core computations in FP16 while keeping critical steps in higher precision; a regularization parameter $\alpha$ stabilizes inner low-precision iterations and backward-forward error analyses establish convergence guarantees. The paper also integrates Gaussian Process Regression to predict the spectral splitting parameter $\alpha$, using FP32-trained training sets to accelerate predictions without sacrificing accuracy. Numerical experiments on 3D convection-diffusion, CARE, and Sylvester problems demonstrate substantial speedups and robust convergence, with $\alpha$-dependent stability effects clearly illustrated. These results highlight the practical potential of combining mixed-precision arithmetic, regularization, and data-driven parameter prediction for efficient, reliable large-scale linear solving in HPC contexts.

Abstract

This study presents a novel mixed-precision iterative refinement algorithm, GADI-IR, within the general alternating-direction implicit (GADI) framework, designed for efficiently solving large-scale sparse linear systems. By employing low-precision arithmetic, particularly half-precision (FP16), for computationally intensive inner iterations, the method achieves substantial acceleration while maintaining high numerical accuracy. Key challenges such as overflow in FP16 and convergence issues for low precision are addressed through careful backward error analysis and the application of a regularization parameter $α$. Furthermore, the integration of low-precision arithmetic into the parameter prediction process, using Gaussian process regression (GPR), significantly reduces computational time without degrading performance. The method is particularly effective for large-scale linear systems arising from discretized partial differential equations and other high-dimensional problems, where both accuracy and efficiency are critical. Numerical experiments demonstrate that the use of FP16 and mixed-precision strategies not only accelerates computation but also ensures robust convergence, making the approach advantageous for various applications. The results highlight the potential of leveraging lower-precision arithmetic to achieve superior computational efficiency in high-performance computing.

Three-precision iterative refinement with parameter regularization and prediction for solving large sparse linear systems

TL;DR

GADI-IR introduces a three-precision iterative refinement strategy within the GADI framework to solve large sparse linear systems Ax = b by performing core computations in FP16 while keeping critical steps in higher precision; a regularization parameter stabilizes inner low-precision iterations and backward-forward error analyses establish convergence guarantees. The paper also integrates Gaussian Process Regression to predict the spectral splitting parameter , using FP32-trained training sets to accelerate predictions without sacrificing accuracy. Numerical experiments on 3D convection-diffusion, CARE, and Sylvester problems demonstrate substantial speedups and robust convergence, with -dependent stability effects clearly illustrated. These results highlight the practical potential of combining mixed-precision arithmetic, regularization, and data-driven parameter prediction for efficient, reliable large-scale linear solving in HPC contexts.

Abstract

This study presents a novel mixed-precision iterative refinement algorithm, GADI-IR, within the general alternating-direction implicit (GADI) framework, designed for efficiently solving large-scale sparse linear systems. By employing low-precision arithmetic, particularly half-precision (FP16), for computationally intensive inner iterations, the method achieves substantial acceleration while maintaining high numerical accuracy. Key challenges such as overflow in FP16 and convergence issues for low precision are addressed through careful backward error analysis and the application of a regularization parameter . Furthermore, the integration of low-precision arithmetic into the parameter prediction process, using Gaussian process regression (GPR), significantly reduces computational time without degrading performance. The method is particularly effective for large-scale linear systems arising from discretized partial differential equations and other high-dimensional problems, where both accuracy and efficiency are critical. Numerical experiments demonstrate that the use of FP16 and mixed-precision strategies not only accelerates computation but also ensures robust convergence, making the approach advantageous for various applications. The results highlight the potential of leveraging lower-precision arithmetic to achieve superior computational efficiency in high-performance computing.
Paper Structure (20 sections, 6 theorems, 70 equations, 8 figures, 5 tables, 1 algorithm)

This paper contains 20 sections, 6 theorems, 70 equations, 8 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Challenges
  4. Contribution
  5. Preliminaries
  6. Error analysis
  7. GADI-IR framework
  8. Forward analysis
  9. Backward analysis
  10. Parameter prediction and regularization
  11. Regularization
  12. Backward analysis for regularization
  13. Parameter prediction
  14. Parameter prediction in GADI
  15. Parameter prediction training set
  16. ...and 5 more sections

Key Result

Theorem 2.1

The GADI framework(gadi) converges to the unique solution $x$ of the linear system $Ax = b$ for any $\alpha > 0$ and $\omega \in [0, 2)$. Furthermore, the spectral radius $\rho(T(\alpha, \omega))$ satisfies: where the iterative matrix $T(\alpha, \omega)$ is defined as: \newlabelthm10

Figures (8)

  • Figure 1: Flow chart of Gaussian Process Regression (GPR) parameters prediction.
  • Figure 1: Relative Residual with different precisions for 3D convection-diffusion equation.
  • Figure 1: Impact of $\alpha$ on convergence residual of 3d convection-diffusion equation.
  • Figure 2: Impact of $\alpha$ on convergence residual of CARE. \newlabelfig:care-alpha-impact0
  • Figure : (a) $r = 0.01$
  • ...and 3 more figures

Theorems & Definitions (9)

  • Theorem 2.1: Convergence of the GADI Framework, Jiang et al., 2022
  • Lemma 2.2: 10.5555/500666
  • Theorem 2.3
  • Proof 1
  • Corollary 2.4
  • Proof 2
  • Lemma 2.5: 10.5555/500666
  • Theorem 2.6
  • Proof 3