A Practical Adaptive Subgame Perfect Gradient Method
Alan Luner, Benjamin Grimmer
TL;DR
ASPGM addresses the practical-speed vs. guarantees dilemma in smooth convex optimization by fusing adaptive, linesearch-free updates with memory-driven subgame-perfect updates. The core BSPGM algorithm provides minimax-optimal guarantees without backtracking, while ASPGM extends it with adaptive restarts and B-based preconditioning to achieve fast convergence on a broad class of problems. Theoretical results establish subgame perfection, dynamic guarantees, and contraction under strong convexity, with empirical benchmarks showing competitive performance against state-of-the-art L-BFGS variants and superior guarantees on many settings. The combination of adaptive conditioning, memory-based optimization, and robust stopping criteria makes ASPGM a practical, certifiable alternative for large-scale smooth convex optimization.
Abstract
We present a performant gradient method for smooth convex optimization, drawing inspiration from several recent advances in the field. Our algorithm, the Adaptive Subgame Perfect Gradient Method (ASPGM) is based on the notion of subgame perfection, attaining a dynamic strengthening of minimax optimality. At each iteration, ASPGM makes a momentum-type update, optimized dynamically based on a (limited) memory/bundle of past first-order information. ASPGM is linesearch-free, parameter-free, and adaptive due to its use of recently developed auto-conditioning, restarting, and preconditioning ideas. We show that ASPGM is competitive with state-of-the-art L-BFGS methods on a wide range of smooth convex problems. Unlike quasi-Newton methods, however, our core algorithm underlying ASPGM has strong, subgame perfect, non-asymptotic guarantees, providing certificates of solution quality, resulting in simple stopping criteria and restarting conditions.