Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Adaptive Cubic Regularization quasi-Newton Method on Riemannian Manifolds

Mauricio S. Louzeiro, Gilson N. Silva, Jinyun Yuan, Daoping Zhang

TL;DR

A quasi-Newton method with cubic regularization is designed for solving Riemannian unconstrained nonconvex optimization problems and remains applicable even in cases of the gradient and Hessian of the objective function unknown.

Abstract

A quasi-Newton method with cubic regularization is designed for solving Riemannian unconstrained nonconvex optimization problems. The proposed algorithm is fully adaptive with at most ${\cal O} (ε_g^{-3/2})$ iterations to achieve a gradient smaller than $ε_g$ for given $ε_g$, and at most $\mathcal O(\max\{ ε_g^{-\frac{3}{2}}, ε_H^{-3} \})$ iterations to reach a second-order stationary point respectively. Notably, the proposed algorithm remains applicable even in cases of the gradient and Hessian of the objective function unknown. Numerical experiments are performed with gradient and Hessian being approximated by forward finite-differences to illustrate the theoretical results and numerical comparison.

An Adaptive Cubic Regularization quasi-Newton Method on Riemannian Manifolds

TL;DR

A quasi-Newton method with cubic regularization is designed for solving Riemannian unconstrained nonconvex optimization problems and remains applicable even in cases of the gradient and Hessian of the objective function unknown.

Abstract

A quasi-Newton method with cubic regularization is designed for solving Riemannian unconstrained nonconvex optimization problems. The proposed algorithm is fully adaptive with at most iterations to achieve a gradient smaller than for given , and at most iterations to reach a second-order stationary point respectively. Notably, the proposed algorithm remains applicable even in cases of the gradient and Hessian of the objective function unknown. Numerical experiments are performed with gradient and Hessian being approximated by forward finite-differences to illustrate the theoretical results and numerical comparison.
Paper Structure (11 sections, 15 theorems, 90 equations, 3 figures, 1 table)

This paper contains 11 sections, 15 theorems, 90 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Preliminary
  3. The Riemannian quasi-Newton Method
  4. Worst-Case Iteration Complexity Analysis
  5. First-order analysis
  6. Second-order analysis
  7. Finite-Difference Approximations for Gradients and Hessians
  8. Approximation for Gradients
  9. Approximations for Hessians
  10. Numerical experiments
  11. Conclusion

Key Result

Lemma 1

Let $f\colon \mathop{\mathrm{\cal M}}\nolimits \to \mathop{\mathrm{\mathbb{R}}}\nolimits$ be twice differentiable on a complete Riemannian manifold $\mathop{\mathrm{\cal M}}\nolimits$. If $f\colon \mathop{\mathrm{\cal M}}\nolimits \to \mathop{\mathrm{\mathbb{R}}}\nolimits$ has an $L$-Lipschitz cont and

Figures (3)

  • Figure 1: Objective function value at each iteration on the six Riemannian optimization problems.
  • Figure 2: Norm of approximated Riemannian gradient at each iteration on the six Riemannian optimization problems.
  • Figure 3: Objective function value and norm of approximated Riemannian gradient at each iteration for minimizing the composite function on a sphere manifold.

Theorems & Definitions (33)

  • Definition 1: absil2008optimization
  • Definition 2: agarwal2021adaptive
  • Lemma 1: agarwal2021adaptive
  • Lemma 2: grapiglia2022cubic
  • Remark 1
  • Remark 2
  • Remark 3
  • Theorem 1
  • proof
  • Corollary 1
  • ...and 23 more