Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mirror Descent on Riemannian Manifolds

Jiaxin Jiang, Lei Shi, Jiyuan Tan

Abstract

Mirror Descent (MD) is a scalable first-order method widely used in large-scale optimization, with applications in image processing, policy optimization, and neural network training. This paper generalizes MD to optimization on Riemannian manifolds. In particular, we develop a Riemannian Mirror Descent (RMD) framework via reparameterization and further propose a stochastic variant of RMD. We also establish non-asymptotic convergence guarantees for both RMD and stochastic RMD. As an application to the Stiefel manifold, our RMD framework reduces to the Curvilinear Gradient Descent (CGD) method proposed in [26]. Moreover, when specializing the stochastic RMD framework to the Stiefel setting, we obtain a stochastic extension of CGD, which effectively addresses large-scale manifold optimization problems.

Mirror Descent on Riemannian Manifolds

Abstract

Mirror Descent (MD) is a scalable first-order method widely used in large-scale optimization, with applications in image processing, policy optimization, and neural network training. This paper generalizes MD to optimization on Riemannian manifolds. In particular, we develop a Riemannian Mirror Descent (RMD) framework via reparameterization and further propose a stochastic variant of RMD. We also establish non-asymptotic convergence guarantees for both RMD and stochastic RMD. As an application to the Stiefel manifold, our RMD framework reduces to the Curvilinear Gradient Descent (CGD) method proposed in [26]. Moreover, when specializing the stochastic RMD framework to the Stiefel setting, we obtain a stochastic extension of CGD, which effectively addresses large-scale manifold optimization problems.
Paper Structure (11 sections, 6 theorems, 77 equations, 2 tables, 2 algorithms)

This paper contains 11 sections, 6 theorems, 77 equations, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Notation and Theoretical Background
  3. Riemannian Mirror Descent: Algorithms and Convergence Theorems
  4. Motivation and Algorithm Framework
  5. Examples of Riemannian Mirror Descent
  6. Stochastic Riemannian Mirror Descent
  7. Convergence Analysis
  8. Numerical Experiments
  9. Linear Eigenvalue Problem
  10. Orthogonal Procrustes Problem
  11. Conclusion

Key Result

Theorem 1

For RMD updates in Algorithm algo:md_rie, assume that Assumptions asp:Phi--asp:bounded_gradient hold. Let $\eta_t\equiv \eta$ be a constant step size satisfying Then the following statements hold:

Theorems & Definitions (23)

  • definition 1
  • definition 2
  • definition 3
  • definition 4
  • definition 5
  • definition 6
  • definition 7
  • definition 8
  • definition 9
  • remark 1
  • ...and 13 more