Accelerated ADMM: Automated Parameter Tuning and Improved Linear Convergence

TL;DR

This work uses the methodology of expressing the accelerated ADMM as a Lur'e system, i.e., an interconnection of a linear dynamical system in feedback with a slope-restricted operator, and uses Integral Quadratic Constraints to establish linear convergence.

Abstract

This work studies the linear convergence of an accelerated scheme of the Alternating Direction Method of Multipliers (ADMM) for strongly convex and Lipschitz-smooth problems. We use the methodology of expressing the accelerated ADMM as a Lur'e system, i.e., an interconnection of a linear dynamical system in feedback with a slope-restricted operator, and we use Integral Quadratic Constraints to establish linear convergence. In addition, we propose several parameter tuning heuristics and their impact on the convergence rate through numerical analyses. Our new bounds show improved linear convergence rates compared to the vanilla algorithm and previous proposed accelerated variants, which is also empirically validated on a LASSO regression benchmark.