Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Finite-Time Analysis of Stochastic Nonconvex Nonsmooth Optimization on the Riemannian Manifolds

Emre Sahinoglu, Youbang Sun, Shahin Shahrampour

TL;DR

The paper tackles finite-time guarantees for nonsmooth, nonconvex stochastic optimization on Riemannian manifolds by adapting Goldstein stationarity to the Riemannian setting and introducing the RO2NC algorithm, with a gradient-free variant ZO-RO2NC. It establishes a provable sample complexity of $O(\delta^{-1}\epsilon^{-3})$ to find $(\delta,\epsilon)$-stationary points, matching Euclidean results, and extends to zeroth-order information while maintaining the same rate. Theoretical development relies on retractions, tangent-space transports, and curvature-aware bounds to control geodesic-based distortions, complemented by numerical experiments on sparse PCA that validate the theory. Collectively, the work provides the first finite-time guarantees for fully nonsmooth nonconvex optimization on manifolds and broadens the practical toolkit for manifold-constrained, nonsmooth learning problems.

Abstract

This work addresses the finite-time analysis of nonsmooth nonconvex stochastic optimization under Riemannian manifold constraints. We adapt the notion of Goldstein stationarity to the Riemannian setting as a performance metric for nonsmooth optimization on manifolds. We then propose a Riemannian Online to NonConvex (RO2NC) algorithm, for which we establish the sample complexity of $O(ε^{-3}δ^{-1})$ in finding $(δ,ε)$-stationary points. This result is the first-ever finite-time guarantee for fully nonsmooth, nonconvex optimization on manifolds and matches the optimal complexity in the Euclidean setting. When gradient information is unavailable, we develop a zeroth order version of RO2NC algorithm (ZO-RO2NC), for which we establish the same sample complexity. The numerical results support the theory and demonstrate the practical effectiveness of the algorithms.

Finite-Time Analysis of Stochastic Nonconvex Nonsmooth Optimization on the Riemannian Manifolds

TL;DR

The paper tackles finite-time guarantees for nonsmooth, nonconvex stochastic optimization on Riemannian manifolds by adapting Goldstein stationarity to the Riemannian setting and introducing the RO2NC algorithm, with a gradient-free variant ZO-RO2NC. It establishes a provable sample complexity of to find -stationary points, matching Euclidean results, and extends to zeroth-order information while maintaining the same rate. Theoretical development relies on retractions, tangent-space transports, and curvature-aware bounds to control geodesic-based distortions, complemented by numerical experiments on sparse PCA that validate the theory. Collectively, the work provides the first finite-time guarantees for fully nonsmooth nonconvex optimization on manifolds and broadens the practical toolkit for manifold-constrained, nonsmooth learning problems.

Abstract

This work addresses the finite-time analysis of nonsmooth nonconvex stochastic optimization under Riemannian manifold constraints. We adapt the notion of Goldstein stationarity to the Riemannian setting as a performance metric for nonsmooth optimization on manifolds. We then propose a Riemannian Online to NonConvex (RO2NC) algorithm, for which we establish the sample complexity of in finding -stationary points. This result is the first-ever finite-time guarantee for fully nonsmooth, nonconvex optimization on manifolds and matches the optimal complexity in the Euclidean setting. When gradient information is unavailable, we develop a zeroth order version of RO2NC algorithm (ZO-RO2NC), for which we establish the same sample complexity. The numerical results support the theory and demonstrate the practical effectiveness of the algorithms.
Paper Structure (27 sections, 11 theorems, 90 equations, 1 figure, 1 table, 1 algorithm)

This paper contains 27 sections, 11 theorems, 90 equations, 1 figure, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Highlights of the Technical Analysis
  3. Literature Review
  4. Preliminaries
  5. Background on Manifold Optimization
  6. Goldstein Stationarity on Riemannian Manifolds
  7. Technical Assumptions
  8. First Order Setting
  9. The Structure of RO2NC versus O2NC
  10. RO2NC with Parallel Transport
  11. RO2NC with Projection
  12. Zeroth Order Setting
  13. Numerical Experiments
  14. Conclusion, Limitations, and Future Work
  15. Proofs
  16. ...and 12 more sections

Key Result

Lemma 2.8

Suppose $\mathcal{M}$ is an embedded submanifold of the Euclidean space with a second fundamental form bounded by $C$. Let $\gamma :[0,t]\to \mathcal{M}$ be a broken geodesic (a piecewise smooth curve with a finite number of curve segments, each of which is a geodesic) and $v \in T_{\gamma(0)} \math where the parallel transport is computed along the path $\gamma$.

Figures (1)

  • Figure 1: Evaluation of gradient norms in both settings; $D$: clipping parameter, $\eta$: step size.

Theorems & Definitions (24)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Lemma 2.8
  • Lemma 2.9
  • Theorem 3.1
  • Remark 3.2
  • Theorem 3.3
  • Remark 3.4
  • Lemma 4.1
  • ...and 14 more