Adaptive Riemannian ADMM for Nonsmooth Optimization: Optimal Complexity without Smoothing
Kangkang Deng, Jiachen Jin, Jiang Hu, Hongxia Wang
TL;DR
This work addresses nonsmooth optimization on compact Riemannian manifolds for a composite objective $f(x) + h(\mathcal A x)$. It introduces Adaptive Riemannian ADMM (ARADMM), which avoids smoothing by adaptively tuning dual stepsizes $\gamma_k$ and penalty parameters $\rho_k$, so that each iteration requires only one Riemannian gradient evaluation and one proximal update. The authors prove an optimal iteration complexity of $O(\epsilon^{-3})$ to obtain an $\epsilon$-approximate KKT point, matching smoothing-based methods while directly handling the original nonsmooth problem. Numerical experiments on sparse PCA and robust subspace recovery demonstrate that ARADMM consistently outperforms state-of-the-art Riemannian ADMM variants in convergence speed and solution quality.
Abstract
We study the problem of minimizing the sum of a smooth function and a nonsmooth convex regularizer over a compact Riemannian submanifold embedded in Euclidean space. By introducing an auxiliary splitting variable, we propose an adaptive Riemannian alternating direction method of multipliers (ARADMM), which, for the first time, achieves convergence without requiring smoothing of the nonsmooth term. Our approach involves only one Riemannian gradient evaluation and one proximal update per iteration. Through careful and adaptive coordination of the stepsizes and penalty parameters, we establish an optimal iteration complexity of order $\mathcal{O}(ε^{-3})$ for finding an $ε$-approximate KKT point, matching the complexity of existing smoothing technique-based Riemannian ADMM methods. Extensive numerical experiments on sparse PCA and robust subspace recovery demonstrate that our ARADMM consistently outperforms state-of-the-art Riemannian ADMM variants in convergence speed and solution quality.