NR-SSOR right preconditioned RRGMRES for arbitrary singular systems and least squares problems

TL;DR

The paper addresses solving singular and inconsistent linear systems where GMRES may fail or degrade, by applying RRGMRES to a right-preconditioned transformed problem $A C A^{ m T} z=b$ with SPD $C$. It proves that using NR-SSOR as the inner right preconditioner yields a symmetric, positive definite $C$ and ensures that the preconditioned matrix $AB$ is range-symmetric, enabling stable least-squares solutions for arbitrary $b$ and rectangular LS problems. The authors introduce AB-RRGMRES with NR-SSOR inner iterations, analyze its convergence properties, and compare it against MINRES-QLP and other baselines across GP, index-2, and underdetermined LS tests, showing improved residuals and efficiency in many challenging cases. This approach has practical impact for discretized PDEs and data-fitting tasks where singular or inconsistent systems frequently arise, offering a robust tool for obtaining reliable least-squares solutions.

Abstract

GMRES is known to determine a least squares solution of $ A x = b $ where $ A \in R^{n \times n} $ without breakdown for arbitrary $ b \in R^n $, and initial iterate $ x_0 \in R^n $ if and only if $ A $ is range-symmetric, i.e. $ R(A^T) = R(A) $, where $ A $ may be singular and $ b $ may not be in the range space $ R(A) $ of $ A $. In this paper, we propose applying the Range Restricted GMRES (RRGMRES) to $ A C A^T z = b $, where $ C \in R^{n \times n} $ is symmetric positive definite. This determines a least squares solution $ x = C A^T z $ of $ A x = b $ without breakdown for arbitrary (singular) matrix $ A \in R^{n \times n} $ and $ b, x_0 \in R^n $, and is much more stable and accurate compared to GMRES, RRGMRES and MINRES-QLP applied to $ A x = b $ for inconsistent problems when $ b \notin R(A) $. In particular, we propose applying the NR-SSOR as the inner iteration right preconditioner, which also works efficiently for least squares problems $ \min_{x \in R^n} \| b - A x\|_2 $ for $ A \in R^{m \times n} $ and arbitrary $ b \in R^m $. Numerical experiments demonstrate the validity of the proposed method.

Figures (8)

• Figure 1: $\frac{\|A^{{\rm T}}\hbox{\boldmath $r$}_{k}\|_{2}}{\|A^{{\rm T}}\hbox{\boldmath $b$}\|_{2}}$ versus the iteration number of RRGMRES for the GP inconsistent system
• Figure 2: $\frac{\|A^{{\rm T}}\hbox{\boldmath $r$}_{k}\|_{2}}{\|A^{{\rm T}}\hbox{\boldmath $b$}\|_{2}}$ versus the iteration number for AB-RRGMRES using NR-SSOR with 1 inner iteration ($\star$), $B=\{{\rm diag}(A^{{\hbox{\scriptsize T}}}A)\}^{-1} A^{\rm T}$ ($+$), and $B=A^{\rm T}$ ($\times$) for the GP inconsistent system
• Figure 3: $\frac{\|A^{{\rm T}}\hbox{\boldmath $r$}_{k}\|_{2}}{\|A^{{\rm T}}\hbox{\boldmath $b$}\|_{2}}$ versus the iteration number of RRGMRES for the index 2 inconsistent system
• Figure 4: $\frac{\|A^{{\rm T}}\hbox{\boldmath $r$}_{k}\|_{2}}{\|A^{{\rm T}}\hbox{\boldmath $b$}\|_{2}}$ versus the iteration number of AB-RRGMRES using NR-SSOR with 1 inner iteration ($\star$), $B=\{{\rm diag}(A^{{\hbox{\scriptsize T}}}A)\}^{-1} A^{\rm T}$ ($+$), and $B=A^{\rm T}$ ($\times$) for the index 2 inconsistent system
• Figure 5: $\frac{\|A^{{\rm T}}\hbox{\boldmath $r$}_{k}\|_{2}}{\|A^{{\rm T}}\hbox{\boldmath $b$}\|_{2}}$ versus the iteration number for MINRES-QLP applied to the GP inconsistent problem ($\star$)

Theorems & Definitions (15)

• Definition 1
• Theorem 2
• proof
• Theorem 3
• Lemma 1
• proof
• Theorem 4
• Lemma 2
• proof
• Theorem 5