Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Modified memoryless spectral-scaling Broyden family on Riemannian manifolds

Hiroyuki Sakai, Hideaki Iiduka

TL;DR

This work introduces a modified memoryless quasi-Newton method for optimization on Riemannian manifolds by augmenting the spectral-scaling Broyden family with an additional parameter and allowing a general transport map. The proposed approach defines a descent-oriented search direction that combines gradient information with transport-adjusted updates and a new scalar parameter, guaranteeing sufficient descent under specified bounds and yielding global convergence under Wolfe-type step conditions. The framework subsumes existing memoryless methods as special cases and enables the use of maps beyond vector transport, such as inverse retractions, enhancing flexibility. Numerical experiments on the unit sphere and the oblique manifold show that particular parameter choices, especially a small $\\xi_{k-1}$ and BFGS-like settings, can outperform existing methods, highlighting practical benefits for large-scale Riemannian optimization. Overall, the paper advances robust, scalable quasi-Newton techniques for manifold optimization with solid theoretical guarantees and favorable empirical performance.

Abstract

This paper presents modified memoryless quasi-Newton methods based on the spectral-scaling Broyden family on Riemannian manifolds. The method involves adding one parameter to the search direction of the memoryless self-scaling Broyden family on the manifold. Moreover, it uses a general map instead of vector transport. This idea has already been proposed within a general framework of Riemannian conjugate gradient methods where one can use vector transport, scaled vector transport, or an inverse retraction. We show that the search direction satisfies the sufficient descent condition under some assumptions on the parameters. In addition, we show global convergence of the proposed method under the Wolfe conditions. We numerically compare it with existing methods, including Riemannian conjugate gradient methods and the memoryless spectral-scaling Broyden family. The numerical results indicate that the proposed method with the BFGS formula is suitable for solving an off-diagonal cost function minimization problem on an oblique manifold.

Modified memoryless spectral-scaling Broyden family on Riemannian manifolds

TL;DR

This work introduces a modified memoryless quasi-Newton method for optimization on Riemannian manifolds by augmenting the spectral-scaling Broyden family with an additional parameter and allowing a general transport map. The proposed approach defines a descent-oriented search direction that combines gradient information with transport-adjusted updates and a new scalar parameter, guaranteeing sufficient descent under specified bounds and yielding global convergence under Wolfe-type step conditions. The framework subsumes existing memoryless methods as special cases and enables the use of maps beyond vector transport, such as inverse retractions, enhancing flexibility. Numerical experiments on the unit sphere and the oblique manifold show that particular parameter choices, especially a small and BFGS-like settings, can outperform existing methods, highlighting practical benefits for large-scale Riemannian optimization. Overall, the paper advances robust, scalable quasi-Newton techniques for manifold optimization with solid theoretical guarantees and favorable empirical performance.

Abstract

This paper presents modified memoryless quasi-Newton methods based on the spectral-scaling Broyden family on Riemannian manifolds. The method involves adding one parameter to the search direction of the memoryless self-scaling Broyden family on the manifold. Moreover, it uses a general map instead of vector transport. This idea has already been proposed within a general framework of Riemannian conjugate gradient methods where one can use vector transport, scaled vector transport, or an inverse retraction. We show that the search direction satisfies the sufficient descent condition under some assumptions on the parameters. In addition, we show global convergence of the proposed method under the Wolfe conditions. We numerically compare it with existing methods, including Riemannian conjugate gradient methods and the memoryless spectral-scaling Broyden family. The numerical results indicate that the proposed method with the BFGS formula is suitable for solving an off-diagonal cost function minimization problem on an oblique manifold.
Paper Structure (8 sections, 3 theorems, 67 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 8 sections, 3 theorems, 67 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Mathematical preliminaries
  3. Memoryless spectral-scaling Broyden family
  4. Memoryless spectral-scaling Broyden family on Riemannian manifolds
  5. Proposed algorithm
  6. Convergence analysis
  7. Numerical experiments
  8. Conclusion

Key Result

Theorem 1

Suppose that Assumptions asm:map and asm:zoutendijk hold. Let $(x_k)_{k=0,1,\cdots}$ be a sequence generated by an iterative method of the form eq:iterative. We assume that the step size $\alpha_k$ satisfies the $\mathscr{T}^{(k)}$-Wolfe conditions eq:armijo and eq:wolfe. If the search direction $\e where $K$ is the subset of $\mathbb{N}$ in Assumption asm:map.

Figures (4)

  • Figure 1: Performance profiles of each algorithm versus the number of iterations (a) and the elapsed time (b) for Problem \ref{['pbl:rayleigh']}. $z_k$ is defined by Li-Fukushima regularization \ref{['eq:li-fukushima-1']} and \ref{['eq:li-fukushima-2']}.
  • Figure 2: Performance profiles of each algorithm versus the number of iterations (a) and the elapsed time (b) for Problem \ref{['pbl:rayleigh']}. $z_k$ is defined by Powell’s damping technique \ref{['eq:powell-1']} and \ref{['eq:powell-2']}.
  • Figure 3: Performance profiles of each algorithm versus the number of iterations (a) and the elapsed time (b) for Problem \ref{['pbl:off-diag']}. $z_k$ is defined by Li-Fukushima regularization \ref{['eq:li-fukushima-1']} and \ref{['eq:li-fukushima-2']}.
  • Figure 4: Performance profiles of each algorithm versus the number of iterations (a) and the elapsed time (b) for Problem \ref{['pbl:off-diag']}. $z_k$ is defined by Powell’s damping technique \ref{['eq:powell-1']} and \ref{['eq:powell-2']}

Theorems & Definitions (7)

  • Definition 1
  • Definition 2
  • Theorem 1
  • Proposition 1
  • proof
  • Theorem 2
  • proof