Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Intrinsic Decentralized Stochastic Riemannian Optimization on Manifolds with Bounded Sectional Curvature

Duc Toan Nguyen, César A. Uribe

Abstract

Decentralized optimization on Riemannian manifolds is foundational for many modern machine learning and signal processing applications in which data are non-Euclidean and generated and processed in a distributed manner. Although intrinsic Riemannian methods exploit manifold geometry without relying on Euclidean embeddings, existing decentralized Riemannian optimization algorithms typically use constant step sizes and therefore converge only to a neighborhood of steady-state error. In this paper, we study the decentralized stochastic Riemannian gradient method in the diminishing step-size regime on manifolds with (possibly positive) bounded sectional curvature. We prove an $O(1/T)$ bound for the network consensus error and an $O(\log T/\sqrt{T})$ ergodic bound for the global optimality gap. To the best of our knowledge, this is the first exact, non-asymptotic optimality-gap guarantee for an intrinsic decentralized stochastic Riemannian method in the geodesically convex setting. Furthermore, the diminishing step-size schedule allows substantially larger initial gradient steps than fixed-step baselines, leading to better performance in practice. We illustrate this on the problem of distributed PCA over a Grassmann manifold.

Intrinsic Decentralized Stochastic Riemannian Optimization on Manifolds with Bounded Sectional Curvature

Abstract

Decentralized optimization on Riemannian manifolds is foundational for many modern machine learning and signal processing applications in which data are non-Euclidean and generated and processed in a distributed manner. Although intrinsic Riemannian methods exploit manifold geometry without relying on Euclidean embeddings, existing decentralized Riemannian optimization algorithms typically use constant step sizes and therefore converge only to a neighborhood of steady-state error. In this paper, we study the decentralized stochastic Riemannian gradient method in the diminishing step-size regime on manifolds with (possibly positive) bounded sectional curvature. We prove an bound for the network consensus error and an ergodic bound for the global optimality gap. To the best of our knowledge, this is the first exact, non-asymptotic optimality-gap guarantee for an intrinsic decentralized stochastic Riemannian method in the geodesically convex setting. Furthermore, the diminishing step-size schedule allows substantially larger initial gradient steps than fixed-step baselines, leading to better performance in practice. We illustrate this on the problem of distributed PCA over a Grassmann manifold.
Paper Structure (8 sections, 7 theorems, 39 equations, 1 figure)

This paper contains 8 sections, 7 theorems, 39 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Algorithm and Results
  3. Convergence Analysis
  4. Numerical Analysis
  5. Experiment set-up
  6. Performance on Erdős-Rényi Graphs
  7. Performance on Cycle Graphs
  8. Discussion and Future Work

Key Result

Theorem 1

Let Assumptions assum:Network-topology, assum:manifold, assum:bounded-grad, and assum:unbiased-estimator hold. Let $\{x_i^t\}_{t=1}^{T+1}$ and $\{y_i^{t}\}_{t=1}^{T+1}$ be generated by eqn:gradient-step--eqn:consensus-step with $x_i^1=x^1\in\mathcal{X}$ for all $i$, $\eta_t={\eta_0}/{\sqrt{t}}$, wit where $C(\xi){=}{(1{+}\xi^2)}/{\xi^4}$, $B{=}(1{-}\xi)\bigl(2C_1(\sigma^2{+}\delta^2){+}\xi^{-1}\de

Figures (1)

  • Figure 1: Consensus and Mean Squared Deviation (MSD) over ER and Cycle graphs with 35, 70, and 100 nodes

Theorems & Definitions (16)

  • Definition 1: Geodesically convex function
  • Definition 2: Fréchet mean
  • Remark 1
  • Theorem 1
  • Remark 2
  • Theorem 2
  • Remark 3
  • Lemma 3: wang2025distributed
  • Lemma 4: sun2019escaping
  • Lemma 5
  • ...and 6 more