Acceleration Meets Inverse Maintenance: Faster $\ell_{\infty}$-Regression

TL;DR

This work tackles the problem of fast $\u221e_{ abla} ext{regression}$ in the $\, ext{$ abla_{ackslash}ty$}$-norm by integrating acceleration techniques with inverse-maintenance. The authors design both randomized and deterministic MWU schemes that combine width-reduction, sketching, and multi-level inverse-maintenance while maintaining stability and robustness properties, enabling efficient lazy updates. The randomized approach achieves $ ilde{O}(n^{2+1/22.5} ext{poly}(1/))$ time and $ ilde{O}(n^{2+1/22.5})$ iterations with $  o 0$, whereas the deterministic variant reaches $ ilde{O}(n^{2+1/12} ext{poly}(1/))$ time and $ ilde{O}(n^{1/3})$ iterations; both improve upon prior $ ilde{O}(n^{2+1/18})$ and $ ilde{O}(n^{2+1/6})$ baselines in the low-accuracy regime. A central contribution is a novel $oldsymbol{}$-stability notion enabling stable width-reduction in a non-monotone MWU and a careful martingale-based analysis of sketching noise, which together yield tight runtime guarantees. The paper also demonstrates the first end-to-end integration of acceleration with inverse maintenance, suggesting a path toward near-optimal $n^{2+o(1)}$ performance for a broad class of structured convex objectives. Overall, these results push forward the practical and theoretical synchronization of acceleration, sketching, and inverse-maintenance in modern convex optimization.

Abstract

We propose a randomized multiplicative weight update (MWU) algorithm for $\ell_{\infty}$ regression that runs in $\widetilde{O}\left(n^{2+1/22.5} \text{poly}(1/ε)\right)$ time when $ω= 2+o(1)$, improving upon the previous best $\widetilde{O}\left(n^{2+1/18} \text{poly} \log(1/ε)\right)$ runtime in the low-accuracy regime. Our algorithm combines state-of-the-art inverse maintenance data structures with acceleration. In order to do so, we propose a novel acceleration scheme for MWU that exhibits {\it stabiliy} and {\it robustness}, which are required for the efficient implementations of the inverse maintenance data structures. We also design a faster {\it deterministic} MWU algorithm that runs in $\widetilde{O}\left(n^{2+1/12}\text{poly}(1/ε)\right))$ time when $ω= 2+o(1)$, improving upon the previous best $\widetilde{O}\left(n^{2+1/6} \text{poly} \log(1/ε)\right)$ runtime in the low-accuracy regime. We achieve this by showing a novel stability result that goes beyond previously known works based on interior point methods (IPMs). Our work is the first to use acceleration and inverse maintenance together efficiently, finally making the two most important building blocks of modern structured convex optimization compatible.

Figures (1)

• Figure 1: Algorithmic techniques and their requirements on our optimizer.

Theorems & Definitions (88)

• Theorem 1.1: Informal statement of Theorem \ref{['thm:time_deterministic']}
• Theorem 1.2: Informal statement of Theorem \ref{['thm:time_combine']}
• Theorem 2.1: chin2013runtime
• Lemma 2.2: One-level inverse maintenance, (Informal) Theorem 4.1 of bns19
• Lemma 2.3: Two-level inverse maintenance, (Informal) Theorem 4.2 of bns19
• Lemma 3.0: Stability bound of $\ell_3$ norm over all primal iterations
• Lemma 3.0: Stability bound of $\ell_3$ norm over all width reduction iterations
• Theorem 3.1
• Lemma A.1
• Lemma A.2