Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed Stochastic Proximal Algorithm on Riemannian Submanifolds for Weakly-convex Functions

Jishu Zhao, Xi Wang, Jinlong Lei, Shixiang Chen

TL;DR

The paper tackles distributed stochastic optimization with manifold constraints by introducing a retraction-based proximal framework that preserves feasibility on compact embedded submanifolds. It analyzes three stochastic algorithms (subgradient, proximal point, proximal linear) and proves that, starting from a local region, iterates reach a nearly stationary point in expectation while achieving consensus, with a convergence rate of $O\left(\frac{1+\kappa_g}{\sqrt{k}}\right)$ that explicitly factors in geodesic curvature $\kappa_g$. The methodology leverages the Moreau envelope, Riemannian subdifferentials, and curvature-aware geometric inequalities to bridge Euclidean and manifold analyses. Applications to distributed blind deconvolution on the sphere, distributed sparse dictionary learning, and generalized eigenvalue problems on generalized Stiefel manifolds demonstrate practical effectiveness and the influence of manifold geometry on performance. The work advances intrinsic distributed optimization on general manifolds, offering a scalable, geometry-respecting alternative to projection-based methods.

Abstract

This paper aims to investigate the distributed stochastic optimization problems on compact embedded submanifolds (in the Euclidean space) for multi-agent network systems. To address the manifold structure, we propose a distributed Riemannian stochastic proximal algorithm framework by utilizing the retraction and Riemannian consensus protocol, and analyze three specific algorithms: the distributed Riemannian stochastic subgradient, proximal point, and prox-linear algorithms. When the local costs are weakly-convex and the initial points satisfy certain conditions, we show that the iterates generated by this framework converge to a nearly stationary point in expectation while achieving consensus. We further establish the convergence rate of the algorithm framework as $\mathcal{O}(\frac{1+κ_g}{\sqrt{k}})$ where $k$ denotes the number of iterations and $κ_g$ shows the impact of manifold geometry on the algorithm performance. Finally, numerical experiments are implemented to demonstrate the theoretical results and show the empirical performance.

Distributed Stochastic Proximal Algorithm on Riemannian Submanifolds for Weakly-convex Functions

TL;DR

The paper tackles distributed stochastic optimization with manifold constraints by introducing a retraction-based proximal framework that preserves feasibility on compact embedded submanifolds. It analyzes three stochastic algorithms (subgradient, proximal point, proximal linear) and proves that, starting from a local region, iterates reach a nearly stationary point in expectation while achieving consensus, with a convergence rate of that explicitly factors in geodesic curvature . The methodology leverages the Moreau envelope, Riemannian subdifferentials, and curvature-aware geometric inequalities to bridge Euclidean and manifold analyses. Applications to distributed blind deconvolution on the sphere, distributed sparse dictionary learning, and generalized eigenvalue problems on generalized Stiefel manifolds demonstrate practical effectiveness and the influence of manifold geometry on performance. The work advances intrinsic distributed optimization on general manifolds, offering a scalable, geometry-respecting alternative to projection-based methods.

Abstract

This paper aims to investigate the distributed stochastic optimization problems on compact embedded submanifolds (in the Euclidean space) for multi-agent network systems. To address the manifold structure, we propose a distributed Riemannian stochastic proximal algorithm framework by utilizing the retraction and Riemannian consensus protocol, and analyze three specific algorithms: the distributed Riemannian stochastic subgradient, proximal point, and prox-linear algorithms. When the local costs are weakly-convex and the initial points satisfy certain conditions, we show that the iterates generated by this framework converge to a nearly stationary point in expectation while achieving consensus. We further establish the convergence rate of the algorithm framework as where denotes the number of iterations and shows the impact of manifold geometry on the algorithm performance. Finally, numerical experiments are implemented to demonstrate the theoretical results and show the empirical performance.
Paper Structure (29 sections, 18 theorems, 97 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 29 sections, 18 theorems, 97 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Main Contributions
  4. Paper Organization
  5. Preliminaries
  6. Riemannian Geometry
  7. Stationarity Measurements for Weakly-convex Functions
  8. Distributed Riemannian Stochastic Proximal Algorithm
  9. Problem Reformulation
  10. The Algorithm Framework
  11. Algorithmic Examples
  12. Main Results
  13. Key Lemmas
  14. Consensus Analysis
  15. Convergence Results in The Local Region
  16. ...and 14 more sections

Key Result

Lemma 1

lee2018introduction Suppose $\mathcal{M}$ is an embedded submanifold of $\mathbb{R}^n$, and $\gamma:[0,1]\mapsto \mathcal{M} \subset \mathbb{R}^n$. Let $\eta$ denote a smooth vector field along $\gamma$ that is everywhere tangent to $\mathcal{M}$. Then, the Levi-Civita connection $\mathcal{D}_{\dot\ where $\Pi(\cdot,\cdot)$ is the second fundamental form on $\mathcal{M}$ and the Euclidean vector d

Figures (4)

  • Figure 1: The geometry expression of Lemma \ref{['nv']}.
  • Figure 2: Blind deconvolution: The performance of the three proximal model-based algorithms under different communication graphs, $c=0.2$ for DR-SSG, $c=0.5$ for DR-SPP and $c =0.4$ for DR-SPL.
  • Figure 3: Dictionary learning: The performance of the three proximal model-based algorithms under different communication graphs, $c=0.5$ for DR-SSG, $c=0.4$ for DR-SPP and DR-SPL.
  • Figure 4: Distributed generalized eigenvalue problem: The comparison of different $\kappa_g$ on DR-SGG and DR-SPP algorithms, respectively.

Theorems & Definitions (33)

  • Lemma 1
  • Definition 1
  • Lemma 2
  • Definition 2
  • Definition 3
  • Remark 1
  • Definition 4
  • Remark 2
  • Remark 3
  • Lemma 3: Normal vector inequality
  • ...and 23 more