Sparse Iterative Solvers Using High-Precision Arithmetic with Quasi Multi-Word Algorithms

TL;DR

This work investigates high-precision sparse linear solving by deploying quasi multi-word arithmetic (QDW and QTW) in Conjugate Gradient solvers, comparing against standard FP64 and full multi-word approaches (DW, TW). It shows that normalization is crucial for convergence of quasi algorithms, but when applied judiciously (e.g., after AXPY per iteration) QDW and QTW can achieve accuracy close to full multi-word methods while reducing runtime, with QTW in particular offering substantial speedups over TW due to its SIMD-friendly design. Across eight SPD matrices and multiple tolerances, QTW often matches or exceeds the performance of DW/QDW with less overhead, while sometimes reducing iterations to convergence. The findings suggest quasi triple-word arithmetic as a practical alternative to traditional high-precision methods for sparse iterative solvers on mainstream CPUs, with potential extensions to preconditioning and lower-precision bases for AI-oriented hardware. The work highlights a viable path to accurate and efficient high-precision CG in environments where FP64 performance is constrained or unsupported.

Abstract

To obtain accurate results in numerical computation, high-precision arithmetic is a straightforward approach. However, most processors lack hardware support for floating-point formats beyond double precision (FP64). Double-word arithmetic (Dekker 1971) extends precision by using standard floating-point operations to represent numbers with twice the mantissa length. Building on this concept, various multi-word arithmetic methods have been proposed to further increase precision by combining additional words. Simplified variants, known as quasi algorithms, have also been introduced, which trade a certain loss of accuracy for reduced computational cost. In this study, we investigate the performance of quasi algorithms for double- and triple-word arithmetic in sparse iterative solvers based on the Conjugate Gradient method, and compare them with both non-quasi algorithms and standard FP64. We evaluate execution time on an x86 processor, the number of iterations to convergence, and solution accuracy. Although quasi algorithms require appropriate normalization to preserve accuracy - without it, convergence cannot be achieved - they can still reduce runtime when normalization is applied correctly, while maintaining accuracy comparable to full multi-word algorithms. In particular, quasi triple-word arithmetic can yield more accurate solutions without significantly increasing execution time relative to double-word arithmetic and its quasi variant. Furthermore, for certain problems, a reduction in iteration count contributes to additional speedup. Thus, quasi triple-word arithmetic can serve as a compelling alternative to conventional double-word arithmetic in sparse iterative solvers.

Figures (5)

• Figure 1: Performance of SpMV in GB/s. Note: The matrix is stored in FP64 in all cases, while the vectors are stored in the format corresponding to the arithmetic used.
• Figure 2: Performance of DOT in GB/s.
• Figure 3: Breakdown of relative execution time to FP64 for 100 iterations.
• Figure 4: Convergence history, plotted every 100 iterations. Solid lines: relative error norm ($||\bm{x}_{k}-\bm{x}^{*}||_{2}/||\bm{x}^{*}||_{2}$); dash-dotted lines: true relative residual norm ($||\bm{b}-\bm{Ax}_k||_{2}/||\bm{b}||_{2}$); dotted lines: relative residual norm ($||\bm{r}_k||_{2}/||\bm{b}||_{2}$).
• Figure 5: Convergence history without normalization. Line styles correspond to those in Fig. \ref{['fig:cg_conv']}.