Global iterative methods for sparse approximate inverses of symmetric positive-definite matrices

TL;DR

This work tackles the challenge of computing sparse approximate inverses for symmetric positive-definite matrices to serve as preconditioners. It introduces global iterative methods—N(P)CG, CG, and LO(P)MR—for SPAI computation, alongside dropping strategies to cap sparsity growth. Empirical results show locally optimal minimal residual (LOMR) methods deliver the most robust and fastest convergence, with CG offering competitive performance and NCG mainly improving over SD but sometimes suffering under dropping or ill-conditioning. The findings demonstrate that, when combined with dropping, LO MR yields high-quality, sparse SPD SPAIs that significantly accelerate preconditioned solves, though some matrices resist SPD SPAIs under strict density caps. The work highlights practical implications for preconditioning in large-scale SPD systems and points to avenues for extending these methods to factorized SPAIs (FSAIs) and parallel implementations.

Abstract

The nonlinear (preconditioned) conjugate gradient N(P)CG method and the locally optimal (preconditioned) minimal residual LO(P)MR method, both of which are used for the iterative computation of sparse approximate inverses (SPAIs) of symmetric positive-definite (SPD) matrices, are introduced and analyzed. The (preconditioned) conjugate gradient (P)CG method is also employed and presented for comparison. The N(P)CG method is defined as a one-dimensional projection with residuals made orthogonal to the current search direction, itself made $A$-orthogonal to the last search direction. The residual orthogonality, expressed via Frobenius inner product, actually holds against all previous search directions, making each iterate globally optimal, that is, that minimizes the Frobenius A-norm of the error over the affine Krylov subspace of $A^2$ generated by the initial gradient. The LO(P)MR method is a two-dimensional projection method that enriches iterates produced by the (preconditioned) minimal residual (P)MR method. These approaches differ from existing descent methods and aim to accelerate convergence compared to other global iteration methods, including (P)MR and (preconditioned) steepest descent (P)SD, previously used for SPAI computation. The methods are implemented with practical dropping strategies to control the growth of nonzero components in the approximate inverse. Numerical experiments are performed in which approximate inverses of several sparse SPD matrices are computed. N(P)CG provides a slight improvement over (P)SD, but remains generally less effective than (P)MR. On the other hand, while (P)CG does improve (P)MR, its convergence is more affected by the dropping of nonzero components, ill-conditioning, and small eigenvalues. LO(P)MR is more robust than (P)MR and (P)CG, consistently outperforms other methods, converging faster to better and often sparser approximations.

Figures (10)

• Figure 1: Nonzero structures of test sparse matrices (Experiment00).
• Figure 2: Convergence plots of dropping-free experiments (Experiment01).
• Figure 3: Converged spectra of dropping-free experiments (Experiment01).
• Figure 4: Convergence plot of dropping-free SPAIs and PCG results for the matrix 4bw100eigs20k2 (Experiment03-04 and Experiment07).
• Figure 5: Convergence plots and PCG results for SPAIs of the matrix bundle1 obtained with dropping of nonzero values for maximum densities of 3% (Experiment05-07).

Theorems & Definitions (20)

• Definition 1: NCG method
• Theorem 1: NCG iterates
• proof : Proof of Theorem \ref{['theo:ncg-iterates']}
• Proposition 2: NCG as a Krylov subspace method
• proof : Proof of Proposition \ref{['prop:ncg-krylov']}
• Theorem 3: Optimality of NCG iterates
• proof : Proof of Theorem \ref{['theo:ncg-optimal']}
• Theorem 4: Convergence of NCG iterates
• proof : Proof of Theorem \ref{['theo:error-bound']}
• Definition 2: CG method