Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Riemannian Gradient Method with Momentum

Filippo Leggio, Diego Scuppa

TL;DR

The numerical evidence confirms the effectiveness and robustness of the proposed approach, which provides a meaningful extension of the recently introduced momentum-based method to Riemannian optimization.

Abstract

In this paper, we consider the problem of minimizing a smooth function on a Riemannian manifold and present a Riemannian gradient method with momentum. The proposed algorithm represents a substantial and nontrivial extension of a recently introduced method for unconstrained optimization. We prove that the algorithm, supported by a safeguarding rule, produces an $ε$-stationary point with a worst-case complexity bound of $\mathcal{O}(ε^{-2})$. Extensive computational experiments on benchmark problems are carried out, comparing the proposed method with state-of-the-art solvers available in the Manopt package. The results demonstrate competitive and often superior performance. Overall, the numerical evidence confirms the effectiveness and robustness of the proposed approach, which provides a meaningful extension of the recently introduced momentum-based method to Riemannian optimization.

Riemannian Gradient Method with Momentum

TL;DR

The numerical evidence confirms the effectiveness and robustness of the proposed approach, which provides a meaningful extension of the recently introduced momentum-based method to Riemannian optimization.

Abstract

In this paper, we consider the problem of minimizing a smooth function on a Riemannian manifold and present a Riemannian gradient method with momentum. The proposed algorithm represents a substantial and nontrivial extension of a recently introduced method for unconstrained optimization. We prove that the algorithm, supported by a safeguarding rule, produces an -stationary point with a worst-case complexity bound of . Extensive computational experiments on benchmark problems are carried out, comparing the proposed method with state-of-the-art solvers available in the Manopt package. The results demonstrate competitive and often superior performance. Overall, the numerical evidence confirms the effectiveness and robustness of the proposed approach, which provides a meaningful extension of the recently introduced momentum-based method to Riemannian optimization.
Paper Structure (10 sections, 2 theorems, 36 equations)

This paper contains 10 sections, 2 theorems, 36 equations.

Table of Contents

  1. Introduction
  2. Preliminary technical results on Riemannian manifolds
  3. A convergent first-order framework
  4. An unconstrained gradient method with momentum
  5. Issues behind the generalization to the Riemannian case
  6. The proposed approach
  7. Practical choice of operator $B_k$
  8. The restart strategy
  9. The convergent algorithm
  10. Experimental results

Key Result

Proposition 3.1

Suppose that Assumptions ass:flow--ass:gradientrelated--ass:assunzione are satisfied (with constants $f_{\mathrm{low}}\in \mathbb R$, $0 < c_1\le c_2$, and $L>0$). Let $\delta\in(0,1)$ and $\gamma\in(0,1)$. Let $\{x_k\}$ and $\{d_k\}$ be the sequences produced by Algorithm alg:gradrel. Assume that $ Then, in the worst case, Algorithm alg:gradrel guarantees: Moreover, $ni_{\epsilon}$ satisfies the

Theorems & Definitions (4)

  • Proposition 3.1
  • Remark 1
  • Remark 2
  • Proposition 6.1