Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Obtaining Pseudo-inverse Solutions With MINRES

Yang Liu, Andre Milzarek, Fred Roosta

TL;DR

This work introduces a simple yet effective minimum-norm refinement (MN refinement) that, when applied to the final MINRES iterate solving $\min_x \|b-Ax\|^2$, yields the Moore-Penrose pseudo-inverse solution $x^{+}=A^{\dagger}b$ in inconsistent or numerically singular problems. It extends the refinement to complex-symmetric systems using the Saunders subspace, and to preconditioned MINRES with singular or PSD preconditioners by formulating the refinement within the associated Krylov subspaces. Theoretical results connect the MN refinement to orthogonal projections onto $A\mathcal{K}_t(A,b)$ (Hermitian) or $\bar{A}\mathcal{S}_t(A,b)$ (complex-symmetric), ensuring that the refined iterate equals $A^{\dagger}b$ under appropriate conditions. Numerical experiments on synthetic matrices, Maxwell PDEs, and large-scale image deblurring demonstrate substantial improvements in recovering pseudo-inverse solutions and practical performance, often rivaling MINRES-QLP with lower overhead. The approach offers a broadly applicable, low-cost correction to MINRES that enhances robustness for ill-posed and inconsistent problems with Hermitian and complex-symmetric structures.

Abstract

The celebrated minimum residual method (MINRES), proposed in the seminal paper of Paige and Saunders, has seen great success and widespread use in solving Hermitian (and complex-symmetric) linear systems. Unless the system is consistent, MINRES is not guaranteed to obtain the pseudo-inverse solution. We propose a novel and remarkably simple minimum-norm refinement (MN refinement) that seamlessly integrates with the final MINRES iteration, enabling us to obtain the minimum-norm solution with negligible additional computational cost. We extend our MN refinement to complex-symmetric systems, building on S.-C. Choi's extension of MINRES for solving these systems. Given the flexibility of MINRES to accommodate singular preconditioners, we further investigate the MN refinement in preconditioned settings that involve singular preconditioners. We also provide numerical experiments to support our analysis and showcase the effects of our MN refinement.

Obtaining Pseudo-inverse Solutions With MINRES

TL;DR

This work introduces a simple yet effective minimum-norm refinement (MN refinement) that, when applied to the final MINRES iterate solving , yields the Moore-Penrose pseudo-inverse solution in inconsistent or numerically singular problems. It extends the refinement to complex-symmetric systems using the Saunders subspace, and to preconditioned MINRES with singular or PSD preconditioners by formulating the refinement within the associated Krylov subspaces. Theoretical results connect the MN refinement to orthogonal projections onto (Hermitian) or (complex-symmetric), ensuring that the refined iterate equals under appropriate conditions. Numerical experiments on synthetic matrices, Maxwell PDEs, and large-scale image deblurring demonstrate substantial improvements in recovering pseudo-inverse solutions and practical performance, often rivaling MINRES-QLP with lower overhead. The approach offers a broadly applicable, low-cost correction to MINRES that enhances robustness for ill-posed and inconsistent problems with Hermitian and complex-symmetric structures.

Abstract

The celebrated minimum residual method (MINRES), proposed in the seminal paper of Paige and Saunders, has seen great success and widespread use in solving Hermitian (and complex-symmetric) linear systems. Unless the system is consistent, MINRES is not guaranteed to obtain the pseudo-inverse solution. We propose a novel and remarkably simple minimum-norm refinement (MN refinement) that seamlessly integrates with the final MINRES iteration, enabling us to obtain the minimum-norm solution with negligible additional computational cost. We extend our MN refinement to complex-symmetric systems, building on S.-C. Choi's extension of MINRES for solving these systems. Given the flexibility of MINRES to accommodate singular preconditioners, we further investigate the MN refinement in preconditioned settings that involve singular preconditioners. We also provide numerical experiments to support our analysis and showcase the effects of our MN refinement.
Paper Structure (14 sections, 22 theorems, 89 equations, 7 figures, 1 table, 3 algorithms)

This paper contains 14 sections, 22 theorems, 89 equations, 7 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Pseudo-inverse solutions in MINRES
  3. Hermitian and skew-Hermitian systems
  4. Complex-symmetric systems
  5. Pseudo-inverse solutions in preconditioned MINRES
  6. Preconditioned Hermitian systems
  7. Preconditioned complex-symmetric systems
  8. Numerical experiments
  9. Pseudo-inverse solutions via MN refinement: Theorems \ref{['thm:MINRES_dagger']} and \ref{['thm:CSMINRES_dagger']}
  10. Numerical solution of PDEs by MN refinement: \ref{['prop:MN-refinement_t']}
  11. Image deblurring by MN refinement: \ref{['thm:MINRES_dagger', 'thm:MN-refinement']}
  12. Conclusions
  13. MINRES for complex-symmetric systems
  14. Preconditioned MINRES for complex-symmetric syst- ems

Key Result

lemma 1

The vector $\mathbf{x}$ is a solution of eq:least_squares if and only if there exists a vector $\mathbf{y} \in \mathbb{C}^{d}$ such that $\mathbf{x} = \mathbf{A}^{\dagger} \mathbf{b} + (\mathbf{I}_{d} - \mathbf{A}^{\dagger} \mathbf{A}) \mathbf{y}$. Furthermore, for any solution $\mathbf{x}$, we have

Figures (7)

  • Figure 1: Verifying \ref{['thm:MINRES_dagger', 'thm:CSMINRES_dagger']}. Relative error of the plain (a) and MN refinement (b) iterates of \ref{['alg:MINRES']} applied to \ref{['eq:least_squares']} with Hermitian (sold blue) and complex-symmetric (dashed magenta) matrices. The relative error is computed with respect to the pseudo-inverse solution $\mathbf{A}^{\dagger} \mathbf{b}$ in each case. The $x$-axis denotes the iteration counter.
  • Figure 2: From top to bottom: the magnitude with orientations, the X-axis components, and the Y-axis components. From left to right: the right-hand-side $\mathbf{f}$, the exact solution $\mathbf{u}_{\textnormal{exact}}$, the final MINRES iterate without MN refinement, and the MN refinement result using \ref{['prop:MN-refinement_t']}. A recorded video demonstrating how the MN refinement results convergence to $\mathbf{u}_{\textnormal{exact}}$ iteratively can be found at https://www.youtube.com/watch?v=ivRa-O9DCMI.
  • Figure 3: From the top left to the bottom right: the original image, the noisy blurred image, the image deblurred with LSQR and with LSMR; the image obtained from \ref{['alg:MINRES']}, and the MN refinement image from \ref{['prop:MN-refinement_t']}, that obtained from MINRES-QLP and its corresponding MN refinement image from \ref{['prop:MN-refinement_t']}. The "wall-clock" time shows the total required seconds for each iterative algorithm to satisfy the inexactness criterion $\| \mathbf{A} \mathbf{r} \| \leq 10^{-5} \| \mathbf{A} \mathbf{b} \|$.
  • Figure 4: Image deblurring with Truncated SVD. "Rank $a$%", "$a$%" represents the rank-ratio $r^2/n^2$, where $r$ represents the rank being used in the Truncated SVD method.
  • Figure 5: Image deblurring using \ref{['alg:sub_pMINRES']} with $\mathbf{S}_1 \in \mathbb{R}^{n^2 \times r_1^2}$. In our naming convention, "Rank $a$% ($b$/$c$ sec)", "$a$%" represents the rank-ratio $r_1^2/n^2$, "$c$" is total wall-clock time of the full deblurring process (including the incomplete QR decomposition to compute $\mathbf{S}_1$), and "$b$" shows the time taken by \ref{['alg:sub_pMINRES']} only. The figures in the bottom row are the MN refinement versions of ones in the top row.
  • ...and 2 more figures

Theorems & Definitions (46)

  • definition 1
  • definition 2
  • lemma 1
  • theorem 1: MN refinement for Hermitian Systems
  • proof
  • remark 1
  • proposition 1
  • proof
  • corollary 1: MN refinement for Skew-Hermitian Systems
  • proof
  • ...and 36 more