A Fast solver for high condition linear systems using randomized stable solutions of its blocks
Suvendu Kar, Murugesan Venkatapathi
TL;DR
This work tackles solving ill-conditioned linear systems $A x=b$ by introducing Regularized Orthogonality and Residue based Block-Kaczmarz (ROR-BK). The method combines regularized block updates $x_{t+1}=x_t+A_{\tau}^T(A_{\tau}A_{\tau}^T+\tilde{\lambda}I)^{-1}(b_{\tau}-A_{\tau}x_t)$ with residue-driven dynamic block construction and a sampling scheme based on effective block orthogonality, achieving fast convergence without explicit preconditioning. A convergence analysis, along with a flexible GMRES variant that uses ROR-BK as an inner preconditioner, is provided. Extensive numerical experiments across sparse and dense, well- and ill-conditioned systems show substantial speedups over state-of-the-art SOBK and TA-ReBlocK-U methods, highlighting ROR-BK’s robustness and potential as a practical pre-solver for larger iterative solvers. The appendix further solidifies the theoretical foundations with initial-solution construction and weighted-least-squares convergence results.
Abstract
We present an enhanced version of the row-based randomized block-Kaczmarz method to solve a linear system of equations. This improvement makes use of a regularization during block updates in the solution, and a dynamic proposal distribution based on the current residue and effective orthogonality between blocks. This improved method provides significant gains in solving high-condition number linear systems that are either sparse, or dense least-squares problems that are significantly over/under determined. Considering the poor generalizability of preconditioners for such problems, it can also serve as a pre-solver for other iterative numerical methods when required, and as an inner iteration in certain types of GMRES solvers for linear systems.