The Golden Ratio Proximal ADMM with Norm Independent Step-Sizes for Separable Convex Optimization
Santanu Soe, V. Vetrivel
TL;DR
This work tackles linearly constrained separable convex optimization by introducing two norm-independent step-size strategies for the Golden-ratio proximal ADMM (GrpADMM). The first strategy uses a decaying primal step-size that locally estimates the operator norm without backtracking, while the second strategy employs an increasing step-size with scaled proximal terms to ensure Fejér monotonicity and convergence. The authors prove global convergence and sublinear ergodic rates for both objective gaps and feasibility residuals, under standard assumptions. Numerical experiments on LASSO, ROF denoising, TV deblurring, and unbalanced optimal transport demonstrate robust performance without explicit norm estimation, highlighting the practical impact for large-scale problems.
Abstract
In this work, we propose two step-size strategies for the Golden-ratio proximal ADMM (GrpADMM) to solve linearly constrained separable convex optimization problems. Both strategies eliminate explicit operator-norm estimates by relying solely on inexpensive local information computed at the current iterate without involving backtracking. However, the key difference is that the second step-size rule allows for recovery from poor initial steps and can increase from iteration to iteration. Under standard assumptions, we prove global iterate convergence and derive sublinear rates for both the objective gap and feasibility residuals. Several numerical experiments confirm the adaptability of the approaches, where accurately computing such parameters can be costly or even infeasible.