Krylov Subspace Acceleration for First-Order Splitting Methods in Convex Quadratic Programming

TL;DR

The paper introduces a Krylov subspace acceleration for first-order methods solving convex quadratic programs, recasting FOM updates as piecewise-affine mappings and applying an Arnoldi-based GMRES-style procedure to accelerate convergence. It leverages active-set decompositions to form local affine operators $G_{\mathcal{J}}$ and $h_{\mathcal{J}}$, and couples this with a safeguarding mechanism to ensure stable progress in a preconditioned proximal-point framework. Compared to Anderson acceleration, the proposed method avoids ill-conditioning in slow-convergence regions and demonstrates superior iteration efficiency, with significant wall-time gains on model predictive control (MPC) and sparse learning benchmarks, particularly at high accuracy. The work provides practical guidance on implementation, including concurrency strategies for sparse linear algebra and a thorough evaluation against AA across large problem sets, highlighting its potential for embedded and real-time optimization tasks.

Abstract

We propose an acceleration scheme for first-order methods (FOMs) for convex quadratic programs (QPs) that is analogous to Anderson acceleration and the Generalized Minimal Residual algorithm for linear systems. We motivate our proposed method from the observation that FOMs applied to QPs typically consist of piecewise-affine operators. We describe our Krylov subspace acceleration scheme, contrasting it with existing Anderson acceleration schemes and showing that it largely avoids the latter's well-known ill-conditioning issues in regions of slow convergence. We demonstrate the performance of our scheme relative to Anderson acceleration using standard collections of problems from model predictive control and statistical learning applications. We show that our method is faster than Anderson acceleration across the board in terms of iteration count, and in many cases in computation time, particularly for optimal control and for problems solved to high accuracy.

Figures (6)

• Figure 1: Histogram of ratio between SpMV ($P x$, $A x$, and $A^\top y$) times with complex/real 64-bit floating point vectors. CSC matrices $P$ and $A$ taken from 108 mpc and sslsq problems with $m, n \leq 100000.0$. (3 samples with ratio smaller than 1 removed.)
• Figure 2: Histogram of ratio between time to solve linear systems in-place with matrix $W$ for one versus two right-hand sides. $W$ as in \ref{['eq:qp-preppm-updates']} with $\rho = 0.1$. CSC matrices $P$ and $A$ taken from 104 mpc and sslsq problems with $m, n \leq 20000.0$.
• Figure 3: Relative time performance profile for tolerance $\epsilon = 10^{-3}$ on subset of mpc problems used in goulartClarabelInteriorpointSolver2024. 49 out of 72 problems were solved by at least one solver.
• Figure 4: Relative time performance profile for tolerance $\epsilon = 10^{-6}$ on subset of mpc problems used in goulartClarabelInteriorpointSolver2024. 46 out of 72 problems were solved by at least one solver.
• Figure 5: Relative time performance profile for tolerance $\epsilon = 10^{-3}$ on subset of sslsq problems with $m, n \leq 20.0 000$. All 32 problems were solved by at least one solver.