Single-loop $\mathcal{O}(ε^{-3})$ stochastic smoothing algorithms for nonsmooth Riemannian optimization
Kangkang Deng, Zheng Peng, Weihe Wu
TL;DR
This work develops two single-loop stochastic smoothing algorithms for nonsmooth optimization on compact Riemannian manifolds. By marrying dynamic smoothing with online momentum-based variance reduction, the Lipschitz case attains the optimal $O(\epsilon^{-3})$ iteration complexity, while the indicator-case achieves a near-optimal $\tilde{O}(\epsilon^{-\,\max\{\theta+2,2\theta\}})$ rate under a mild error bound with parameter $\theta\ge1$. Both methods operate online with $O(1)$ samples per iteration and do not rely on nested loops, making them scalable for large-scale problems. The framework unifies smooth stochastic Riemannian optimization, Euclidean composite settings, and constrained optimization, recovering or matching best-known rates across these regimes.
Abstract
In this paper, we develop two Riemannian stochastic smoothing algorithms for nonsmooth optimization problems on Riemannian manifolds, addressing distinct forms of the nonsmooth term \( h \). Both methods combine dynamic smoothing with a momentum-based variance reduction scheme in a fully online manner. When \( h \) is Lipschitz continuous, we propose an stochastic algorithm under adaptive parameter that achieves the optimal iteration complexity of \( \mathcal{O}(ε^{-3}) \), improving upon the best-known rates for exist algorithms. When \( h \) is the indicator function of a convex set, we design a new algorithm using truncated momentum, and under a mild error bound condition with parameter \( θ\geq 1 \), we establish a complexity of \( \tilde{\mathcal{O}}(ε^{-\max\{θ+2, 2θ\}}) \), in line with the best-known results in the Euclidean setting. Both algorithms feature a single-loop design with low per-iteration cost and require only \( \mathcal{O}(1) \) samples per iteration, ensuring that sample and iteration complexities coincide. Our framework encompasses a broad class of problems and recovers or matches optimal complexity guarantees in several important settings, including smooth stochastic Riemannian optimization, composite problems in Euclidean space, and constrained optimization via indicator functions.