ALiA: Adaptive Linearized ADMM
Uijeong Jang, Kaizhao Sun, Wotao Yin, Ernest K Ryu
TL;DR
ALiA is a variant of function-linearized proximal ADMM, which generalizes the classical ADMM by leveraging the differentiable structure of the objective function, making it highly versatile and establishing point convergence of ALiA for convex and differentiable objectives.
Abstract
We propose ALiA, a novel adaptive variant of the alternating direction method of multipliers (ADMM). Specifically, ALiA is a variant of function-linearized proximal ADMM (FLiP ADMM), which generalizes the classical ADMM by leveraging the differentiable structure of the objective function, making it highly versatile. Notably, ALiA features an adaptive stepsize selection scheme that eliminates the need for backtracking linesearch. Motivated by recent advances in adaptive gradient and proximal methods, we establish point convergence of ALiA for convex and differentiable objectives. Furthermore, by introducing negligible computational overhead, we develop an alternative stepsize selection scheme for ALiA that improves the convergence speed both theoretically and empirically. Extensive numerical experiments on practical datasets confirm the accelerated performance of ALiA compared to standard FLiP ADMM. Additionally, we demonstrate that ALiA either outperforms or matches the practical performance of existing adaptive methods across problem classes where it is applicable.