Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Loopless Proximal Riemannian Gradient EXTRA for Distributed Optimization on Compact Manifolds

Yongyang Xiong, Chen Ouyang, Keyou You, Yang Shi, Ligang Wu

TL;DR

Theoretical analysis shows that with a constant stepsize, PR-EXTRA achieves a sublinear convergence rate of $\mathcal{O}(1/K)$ to a stationary point, matching the proximal gradient EXTRA algorithm in Euclidean spaces.

Abstract

Distributed optimization has gained substantial interest in recent years due to its wide applications in machine learning. However, most of existing algorithms are designed for Euclidean spaces, leaving composite optimization on Riemannian manifolds largely unexplored. To bridge this gap, we propose the proximal Riemannian gradient EXTRA algorithm (PR-EXTRA) to solve distributed composite optimization problem with nonsmooth regularizer over compact manifolds. In each iteration, PR-EXTRA requires only a single round communication, coupled with local gradient evaluations and proximal mappings. Furthermore, a manifold projection operator is integrated to ensure the feasibility of all iterates throughout the optimization process. Theoretical analysis shows that with a constant stepsize, PR-EXTRA achieves a sublinear convergence rate of $\mathcal{O}(1/K)$ to a stationary point, matching the proximal gradient EXTRA algorithm in Euclidean spaces. Numerical experiments show the effectiveness of the proposed algorithm.

Loopless Proximal Riemannian Gradient EXTRA for Distributed Optimization on Compact Manifolds

TL;DR

Theoretical analysis shows that with a constant stepsize, PR-EXTRA achieves a sublinear convergence rate of to a stationary point, matching the proximal gradient EXTRA algorithm in Euclidean spaces.

Abstract

Distributed optimization has gained substantial interest in recent years due to its wide applications in machine learning. However, most of existing algorithms are designed for Euclidean spaces, leaving composite optimization on Riemannian manifolds largely unexplored. To bridge this gap, we propose the proximal Riemannian gradient EXTRA algorithm (PR-EXTRA) to solve distributed composite optimization problem with nonsmooth regularizer over compact manifolds. In each iteration, PR-EXTRA requires only a single round communication, coupled with local gradient evaluations and proximal mappings. Furthermore, a manifold projection operator is integrated to ensure the feasibility of all iterates throughout the optimization process. Theoretical analysis shows that with a constant stepsize, PR-EXTRA achieves a sublinear convergence rate of to a stationary point, matching the proximal gradient EXTRA algorithm in Euclidean spaces. Numerical experiments show the effectiveness of the proposed algorithm.
Paper Structure (20 sections, 11 theorems, 94 equations, 6 figures, 1 algorithm)

This paper contains 20 sections, 11 theorems, 94 equations, 6 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries and Problem Formulation
  3. Graph Theory
  4. Problem Formulation
  5. Algorithm Development
  6. Proximal gradient EXTRA Algorithm
  7. Proximal Riemannian gradient EXTRA Algorithm
  8. Convergence Analysis
  9. Numerical Experiments
  10. SPCA Problem
  11. CISE Problem
  12. Conclusion
  13. Proof of Lemma \ref{['l4']}
  14. Proof of Lemma \ref{['l5']}
  15. Proof of Lemma \ref{['A2']}
  16. ...and 5 more sections

Key Result

Lemma 4.1

Given an $R$-proximally smooth compact submanifold $\mathcal{M}$, for any $x \in \mathcal{M}$, $u \in \{ u \in \mathbb{R}^{d\times r} : \|u\| \le \frac{R}{2} \}$, there exists a constant $Q > 0$ such that

Figures (6)

  • Figure 1: Numerical comparison of DR-ProxGT, DRSM, and PR-EXTRA on the SPCA Problem. The up and down figures depict stationarity violations and consensus errors, respectively.
  • Figure 2: Numerical comparison of DR-ProxGT, DRSM, and PR-EXTRA on the CISE Problem. The up and down figures depict stationarity violations and consensus errors, respectively.
  • Figure :
  • Figure :
  • Figure :
  • ...and 1 more figures

Theorems & Definitions (15)

  • Definition 2.1
  • Definition 2.2
  • Remark 3.1
  • Lemma 4.1: Deng2025
  • Lemma 4.2: Chen2024
  • Lemma 4.3: Deng2025
  • Lemma 4.4
  • Lemma 4.5
  • Lemma 4.6
  • Lemma 4.7
  • ...and 5 more