An Empirical Study of Conjugate Gradient Preconditioners for Solving Symmetric Positive Definite Systems of Linear Equations

TL;DR

This study provides a broad empirical benchmark of preconditioned CG solvers for SPD systems, evaluating 10 preconditioners across 108 configurations on 79 SPD matrices to gauge practical performance independent of preconditioner setup costs. It finds that incomplete symmetric factorizations (notably IC and MIC) most often yield the largest reductions in work, with algebraic multigrid offering strong robustness on diagonally dominant cases, while simple classical methods fare poorly. The results underscore the critical roles of matrix ordering (especially AMD) and the inclusion of preconditioner construction costs in assessing true efficiency, showing that per-matrix tuning can substantially improve outcomes at the expense of setup effort. Overall, IC/MIC and AMG emerge as the most effective strategies for SPD preconditioning, though reliability and scalability considerations (e.g., build costs and failures) motivate further study and tuning for large-scale problems. The work has practical implications for selecting and configuring preconditioners in real-world SPD solves, and the authors plan to extend the benchmarks to larger problems and additional ordering strategies.

Abstract

Despite hundreds of papers on preconditioned linear systems of equations, there remains a significant lack of comprehensive performance benchmarks comparing various preconditioners for solving symmetric positive definite (SPD) systems. In this paper, we present a comparative study of 79 matrices using a broad range of preconditioners. Specifically, we evaluate 10 widely used preconditoners across 108 configurations to assess their relative performance against using no preconditioner. Our focus is on preconditioners that are commonly used in practice, are available in major software packages, and can be utilized as black-box tools without requiring significant \textit{a priori} knowledge. In addition, we compare these against a selection of classical methods. We primarily compare them without regards to effort needed to compute the preconditioner. Our results show that symmetric positive definite systems are mostly likely to benefit from incomplete symmetric factorizations, such as incomplete Cholesky (IC). Multigrid methods occasionally do exceptionally well. Simple classical techniques, symmetric Gauss Seidel and symmetric SOR, are not productive. We find that including preconditioner construction costs significantly diminishes the advantages of iterative methods compared to direct solvers; although, tuned IC methods often still outperform direct methods. Additionally, ordering strategies such as approximate minimum degree significantly enhance IC effectiveness. We plan to expand the benchmark with larger matrices, additional solvers, and detailed metrics to provide actionable information on SPD preconditioning.

Figures (95)

• Figure 1: Convergence of the PCG method with various preconditioners applied to the crankseg_1 matrix (53k rows, 10m non-zeros, condition number $2 \cdot 10^4$). The plots have a log-log scale. The control case (diagonal scaling only) is shown in blue for reference, the preconditioner of focus is highlighted in red, and all other preconditioners are shown in gray.
• Figure 2: Convergence of the PCG method with various preconditioners applied to the ecology2 matrix (1m rows, 5m non-zeros, condition number $6 \cdot 10^7$). The plots have a log-log scale. The control case (diagonal scaling only) is shown in blue for reference, the preconditioner of focus is highlighted in red, and all other preconditioners are shown in gray.
• Figure 3: Performance profiles of the PCG method with various preconditioners applied to all matrices in the test set. The red lines show the behavior of multiple configurations of the same preconditioner; the grey lines show the entirely of the results. These show that the IC and MIC have the highest aggregate behavior, but only succeed in slightly over 75% of problems.
• Figure 4: The single-best configuration selects the best configuration over all matrices whereas the tuned-best configuration selects the best run for each matrix within a given preconditioner group (log‐linear). While there are differences, the single best configuration retains the majority of the performance across a wide range of problems.
• Figure 5: Including generation work. Performance profiles of the PCG method with various preconditioners applied to all matrices in the test set where work includes an approximation to the amount of work required to generate the preconditioner.