Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Neural Preconditioners for Iterative Solutions of Sparse Linear Systems

Jie Chen

TL;DR

This paper introduces Graph Neural Preconditioners (GNPs) for accelerating iterative solutions of large sparse linear systems by learning a nonlinear matrix inverse approximation $M oughly A^{-1}$ with a scale-equivariant graph neural network. It integrates $M$ into Flexible GMRES to handle nonlinear preconditioning and provides a convergence analysis, a data-generation strategy that targets the bottom spectrum, and a broad SuiteSparse-based evaluation showing strong robustness, predictable construction times, and competitive execution times relative to ILU, AMG, and GMRES on over 800 matrices. Key contributions include the convergence analysis for FGMRES with nonlinear preconditioning, a practical training-data strategy, and a scale-equivariant GNN design that remains effective across diverse matrices. The results suggest that GNPs offer a viable, robust alternative for general-purpose preconditioning with substantial practical impact across domains with ill-conditioned sparse systems.

Abstract

Preconditioning is at the heart of iterative solutions of large, sparse linear systems of equations in scientific disciplines. Several algebraic approaches, which access no information beyond the matrix itself, are widely studied and used, but ill-conditioned matrices remain very challenging. We take a machine learning approach and propose using graph neural networks as a general-purpose preconditioner. They show attractive performance for many problems and can be used when the mainstream preconditioners perform poorly. Empirical evaluation on over 800 matrices suggests that the construction time of these graph neural preconditioners (GNPs) is more predictable and can be much shorter than that of other widely used ones, such as ILU and AMG, while the execution time is faster than using a Krylov method as the preconditioner, such as in inner-outer GMRES. GNPs have a strong potential for solving large-scale, challenging algebraic problems arising from not only partial differential equations, but also economics, statistics, graph, and optimization, to name a few.

Graph Neural Preconditioners for Iterative Solutions of Sparse Linear Systems

TL;DR

This paper introduces Graph Neural Preconditioners (GNPs) for accelerating iterative solutions of large sparse linear systems by learning a nonlinear matrix inverse approximation with a scale-equivariant graph neural network. It integrates into Flexible GMRES to handle nonlinear preconditioning and provides a convergence analysis, a data-generation strategy that targets the bottom spectrum, and a broad SuiteSparse-based evaluation showing strong robustness, predictable construction times, and competitive execution times relative to ILU, AMG, and GMRES on over 800 matrices. Key contributions include the convergence analysis for FGMRES with nonlinear preconditioning, a practical training-data strategy, and a scale-equivariant GNN design that remains effective across diverse matrices. The results suggest that GNPs offer a viable, robust alternative for general-purpose preconditioning with substantial practical impact across domains with ill-conditioned sparse systems.

Abstract

Preconditioning is at the heart of iterative solutions of large, sparse linear systems of equations in scientific disciplines. Several algebraic approaches, which access no information beyond the matrix itself, are widely studied and used, but ill-conditioned matrices remain very challenging. We take a machine learning approach and propose using graph neural networks as a general-purpose preconditioner. They show attractive performance for many problems and can be used when the mainstream preconditioners perform poorly. Empirical evaluation on over 800 matrices suggests that the construction time of these graph neural preconditioners (GNPs) is more predictable and can be much shorter than that of other widely used ones, such as ILU and AMG, while the execution time is faster than using a Krylov method as the preconditioner, such as in inner-outer GMRES. GNPs have a strong potential for solving large-scale, challenging algebraic problems arising from not only partial differential equations, but also economics, statistics, graph, and optimization, to name a few.
Paper Structure (22 sections, 2 theorems, 16 equations, 18 figures, 4 tables, 1 algorithm)

This paper contains 22 sections, 2 theorems, 16 equations, 18 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. Method
  5. Flexible GMRES
  6. Training Data Generation
  7. Scale-Equivariant Graph Neural Preconditioner
  8. Evaluation Methodology and Experiment Setting
  9. Results
  10. Discussions and Conclusions
  11. Supporting Code
  12. FGMRES
  13. Proofs
  14. Approximately Exponential Convergence of $\|\mathbf{r}_m\|_2$
  15. Scalability
  16. ...and 7 more sections

Key Result

Theorem 1

Assume that FGMRES is run without restart and without breakdown. We use the tilde notation to denote the counterpart quantities when FGMRES is run with a fixed preconditioner matrix $\widetilde{\mathbf{M}}$. For any $\widetilde{\mathbf{M}}$ such that $\mathbf{A} \widetilde{\mathbf{M}}$ can be diagon where $\kappa_2$ denotes the 2-norm condition number, $\mathbf{Q}_m$ (resp. $\widetilde{\mathbf{Q}}

Figures (18)

  • Figure 1: Our GNN architecture for the preconditioner $\mathbf{M}$. The nonlinearity $\sigma$ is $\mathop{\mathrm{ReLU}}\nolimits$. The component $s$ denotes the scale-equivariant operator \ref{['eqn:s']}.
  • Figure 2: Percentage of problems on which each preconditioner performs the best.
  • Figure 3: Preconditioner construction time and solution time (using maxiters to stop). The construction time of Jacobi is negligible and not shown. GMRES does not require construction.
  • Figure 4: Breakdown of best preconditioners with respect to matrix sizes, condition numbers, and application areas. Only the application areas with the top number of problems are shown. The last bar in the middle plot is for condition number $\ge10^{16}$.
  • Figure 5: Example convergence histories.
  • ...and 13 more figures

Theorems & Definitions (4)

  • Theorem 1
  • Corollary 2
  • proof : Proof of Theorem \ref{['thm:fgmres']}
  • proof : Proof of Corollary \ref{['cor:fgmres']}