Optimal Preconditioning is a Geodesically Convex Optimization Problem
M. Levent Doğan, Alperen Ergür, Elias Tsigaridas
TL;DR
The paper presents a unified, geometry-aware approach to preconditioning by recasting condition-number reduction as a geodesically convex optimization over matrix-group actions. It demonstrates that minimizing log condition numbers with respect to unitarily invariant norms yields globally convergent first-order algorithms on symmetric Lie groups, with explicit gradient expressions and convergence guarantees. The framework encompasses both linear and nonlinear (polynomial) systems, providing Frobenius-norm surrogates, shuffling-and-change-of-variables techniques, and clear pathways to practical preconditioners such as diagonal and block-diagonal forms. Empirical results on sparse and dense matrices validate substantial conditioning improvements and show strong correlations between Frobenius-based improvements and Euclidean conditioning, highlighting practical impact for accelerated solvers and robust polynomial system solving.
Abstract
We introduce a unified framework for computing approximately-optimal preconditioners for solving linear and non-linear systems of equations. We demonstrate that the condition number minimization problem, under structured transformations such as diagonal and block-diagonal preconditioners, is geodesically convex with respect to unitarily invariant norms, including the Frobenius and Bombieri--Weyl norms. This allows us to introduce efficient first-order algorithms with precise convergence guarantees. For linear systems, we analyze the action of symmetric Lie subgroups $G \subseteq \GL_m(\CC) \times \GL_n(\CC)$ on the input matrix and prove that the logarithm of the condition number is a smooth geodesically convex function on the associated Riemannian quotient manifold. We obtain explicit gradient formulas, show Lipschitz continuity, and prove convergence rates for computing the optimal Frobenius condition number: $\widetilde{O}(1/\eps^2)$ iterations for general two-sided preconditioners and $\widetilde{O}(κ_F^2 \log(1/\eps))$ for strongly convex cases such as left preconditioning. We extend our framework to consider preconditioning of polynomial systems $\f(x) = 0$, where $\f$ is a system of multivariate polynomials. We analyze the local condition number $μ(\f, ξ)$, at a root $ξ$ and prove that it also admits a geodesically convex formulation under appropriate group actions. We deduce explicit formulas for the Riemannian gradients and present convergence bounds for the corresponding optimization algorithms. To the best of our knowledge, this is the first preconditioning algorithm with theoretical guarantees for polynomial systems.