Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NeuraLSP: An Efficient and Rigorous Neural Left Singular Subspace Preconditioner for Conjugate Gradient Methods

Alexander Benanti, Xi Han, Hong Qin

TL;DR

This work introduces NeuraLSP, a neural left singular subspace preconditioner for conjugate gradient methods that builds a fixed low-rank prolongation operator from the left singular subspace of near-nullspace vectors. By replacing per-instance SVD with a learnable NLSS loss, the method provably recovers the leading left singular subspace and yields a durable coarse space that is robust to rank inflation. Empirically, NeuraLSP delivers substantial speedups (up to 53%) over classical and neural baselines across diffusion, anisotropic, and screened Poisson PDEs, while maintaining convergence behavior. The approach provides a theory-grounded, computationally efficient pathway for integrating spectral information into neural preconditioners for large-scale SPD linear systems.

Abstract

Numerical techniques for solving partial differential equations (PDEs) are integral for many fields across science and engineering. Such techniques usually involve solving large, sparse linear systems, where preconditioning methods are critical. In recent years, neural methods, particularly graph neural networks (GNNs), have demonstrated their potential through accelerated convergence. Nonetheless, to extract connective structures, existing techniques aggregate discretized system matrices into graphs, and suffer from rank inflation and a suboptimal convergence rate. In this paper, we articulate NeuraLSP, a novel neural preconditioner combined with a novel loss metric that leverages the left singular subspace of the system matrix's near-nullspace vectors. By compressing spectral information into a fixed low-rank operator, our method exhibits both theoretical guarantees and empirical robustness to rank inflation, affording up to a 53% speedup. Besides the theoretical guarantees for our newly-formulated loss function, our comprehensive experimental results across diverse families of PDEs also substantiate the aforementioned theoretical advances.

NeuraLSP: An Efficient and Rigorous Neural Left Singular Subspace Preconditioner for Conjugate Gradient Methods

TL;DR

This work introduces NeuraLSP, a neural left singular subspace preconditioner for conjugate gradient methods that builds a fixed low-rank prolongation operator from the left singular subspace of near-nullspace vectors. By replacing per-instance SVD with a learnable NLSS loss, the method provably recovers the leading left singular subspace and yields a durable coarse space that is robust to rank inflation. Empirically, NeuraLSP delivers substantial speedups (up to 53%) over classical and neural baselines across diffusion, anisotropic, and screened Poisson PDEs, while maintaining convergence behavior. The approach provides a theory-grounded, computationally efficient pathway for integrating spectral information into neural preconditioners for large-scale SPD linear systems.

Abstract

Numerical techniques for solving partial differential equations (PDEs) are integral for many fields across science and engineering. Such techniques usually involve solving large, sparse linear systems, where preconditioning methods are critical. In recent years, neural methods, particularly graph neural networks (GNNs), have demonstrated their potential through accelerated convergence. Nonetheless, to extract connective structures, existing techniques aggregate discretized system matrices into graphs, and suffer from rank inflation and a suboptimal convergence rate. In this paper, we articulate NeuraLSP, a novel neural preconditioner combined with a novel loss metric that leverages the left singular subspace of the system matrix's near-nullspace vectors. By compressing spectral information into a fixed low-rank operator, our method exhibits both theoretical guarantees and empirical robustness to rank inflation, affording up to a 53% speedup. Besides the theoretical guarantees for our newly-formulated loss function, our comprehensive experimental results across diverse families of PDEs also substantiate the aforementioned theoretical advances.
Paper Structure (23 sections, 3 theorems, 27 equations, 6 figures, 7 tables, 1 algorithm)

This paper contains 23 sections, 3 theorems, 27 equations, 6 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Mathematical Preliminaries
  4. Linear Systems and PDEs
  5. AMG and Smoothed Aggregation
  6. Near-Nullspace Vectors
  7. SVD for Constructing $\mathbf{P}$
  8. Novel Method
  9. NLSS Loss with Theoretical Guarantees
  10. Neural Network Design
  11. Experiments
  12. System Setup
  13. Diffusion Equation
  14. Anisotropic Equation
  15. Screened Poisson Equation
  16. ...and 8 more sections

Key Result

Theorem 4.1

Let $\mathbf{S}\in\mathbb{R}^{n\times m}$ and let $\tilde{\mathbf{P}} = [\tilde{\mathbf{p_1}}, \cdots, \tilde{\mathbf{p_k}}] \in \text{St}(n,k)$ be a matrix with orthonormal columns ($\tilde{\mathbf{P}}^T\tilde{\mathbf{P}} = \mathbf{I_k}$) where $\text{St}(n,k)$ is the Stiefel manifold of orthonorma

Figures (6)

  • Figure 1: A visual demonstration of how our prolongation matrix can be adjusted based on how much spectral information we decide to capture.
  • Figure 2: Our NeuraLSP pipeline and architecture overview. The system matrix $\mathbf{A}$ is smoothed into a collection of vectors $\mathbf{S}$, which is passed through a neural network consisting of a 4-layer MLP (widths 128-256-256-128) with LayerNorms and GELU activations in-between. Then, the output is projected onto a matrix, and QR-decomposed to ensure orthonormality. The result is the left singular subspace $\tilde{\mathbf{U}}$.
  • Figure 3: Difference from SVD of captured energy of NLSS vs. subspace loss for FEM diffusion equation \ref{['diffusion']}.
  • Figure 4: Difference from SVD of captured energy of NLSS vs. subspace loss for FEM anisotropic equation \ref{['anisotropic']}.
  • Figure 5: Difference from SVD of captured energy of NLSS vs. subspace loss for FEM screened Poisson equation \ref{['screened_poisson']}.
  • ...and 1 more figures

Theorems & Definitions (6)

  • Theorem 4.1
  • Corollary 4.2
  • proof
  • proof
  • Theorem 1.1
  • proof