Revisit First-order Methods for Geodesically Convex Optimization
Yunlu Shu, Jiaxin Jiang, Lei Shi, Tianyu Wang
TL;DR
The paper addresses geodesically convex optimization on Hadamard manifolds and the limitations of prior first-order analyses that rely on curvature bounds and diameter restrictions. It introduces quasilinearization as a curvature-free geometric framework, defining a quasilinearized inner product for geodesic segments and leveraging it to analyze optimization algorithms. The main results establish state-of-the-art convergence rates for proximal-gradient methods: $f(x_t) - f(x^*) = O(1/t)$ deterministically and $\widetilde{O}(1/\sqrt{t})$ stochastically, without the classical A1/A2 assumptions. The framework suggests broad applicability to non-Euclidean optimization, with potential extensions to nonsmooth, variance-reduced, and accelerated schemes.
Abstract
In a seminal work of Zhang and Sra, gradient descent methods for geodesically convex optimization were comprehensively studied. In particular, Zhang and Sra derived a comparison inequality that relates the iterative points in the optimization process. Since their seminal work, numerous follow-ups have studied different downstream usages of their comparison lemma. In this work, we introduce the concept of quasilinearization to optimization, presenting a novel framework for analyzing geodesically convex optimization. By leveraging this technique, we establish state-of-the-art convergence rates -- for both deterministic and stochastic settings -- under weaker assumptions than previously required. The technique of quasilinearization may prove valuable for other non-Euclidean optimization problems.