Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Riemannian multigrid line search for low-rank problems

Marco Sutti, Bart Vandereycken

TL;DR

This work develops Riemannian multigrid line search (RMGLS) for large-scale variational problems that admit low-rank representations. By optimizing directly on the fixed-rank matrix manifold ${\mathcal{M}}_{k}$ and integrating a multilevel full-approximation scheme, the method achieves near mesh-independent convergence with fixed rank throughout iterations. A key contribution is extending the Hager--Zhang line search to the Riemannian setting, enabling machine-precision accuracy despite first-order information. Numerical experiments on linear and nonlinear PDE-inspired problems demonstrate robust performance, effective rank-adaptivity, and clear advantages over Euclidean multilevel and fixed-rank alternatives. The approach offers scalable, accurate low-rank solutions for high-dimensional variational problems arising from PDE discretizations.

Abstract

Large-scale optimization problems arising from the discretization of problems involving PDEs sometimes admit solutions that can be well approximated by low-rank matrices. In this paper, we will exploit this low-rank approximation property by solving the optimization problem directly over the set of low-rank matrices. In particular, we introduce a new multilevel algorithm, where the optimization variable is constrained to the Riemannian manifold of fixed-rank matrices. In contrast to most other multilevel algorithms where the rank is chosen adaptively on each level in order to control the perturbation due to the low-rank truncation, we can keep the ranks (and thus the computational complexity) fixed throughout the iterations. Furthermore, classical implementations of line searches based on Wolfe conditions enable computing a solution where the numerical accuracy is limited to about the square root of the machine epsilon. Here, we propose an extension to Riemannian manifolds of the line search of Hager and Zhang, which uses approximate Wolfe conditions that enable computing a solution on the order of the machine epsilon. Numerical experiments demonstrate the computational efficiency of the proposed framework.

Riemannian multigrid line search for low-rank problems

TL;DR

This work develops Riemannian multigrid line search (RMGLS) for large-scale variational problems that admit low-rank representations. By optimizing directly on the fixed-rank matrix manifold and integrating a multilevel full-approximation scheme, the method achieves near mesh-independent convergence with fixed rank throughout iterations. A key contribution is extending the Hager--Zhang line search to the Riemannian setting, enabling machine-precision accuracy despite first-order information. Numerical experiments on linear and nonlinear PDE-inspired problems demonstrate robust performance, effective rank-adaptivity, and clear advantages over Euclidean multilevel and fixed-rank alternatives. The approach offers scalable, accurate low-rank solutions for high-dimensional variational problems arising from PDE discretizations.

Abstract

Large-scale optimization problems arising from the discretization of problems involving PDEs sometimes admit solutions that can be well approximated by low-rank matrices. In this paper, we will exploit this low-rank approximation property by solving the optimization problem directly over the set of low-rank matrices. In particular, we introduce a new multilevel algorithm, where the optimization variable is constrained to the Riemannian manifold of fixed-rank matrices. In contrast to most other multilevel algorithms where the rank is chosen adaptively on each level in order to control the perturbation due to the low-rank truncation, we can keep the ranks (and thus the computational complexity) fixed throughout the iterations. Furthermore, classical implementations of line searches based on Wolfe conditions enable computing a solution where the numerical accuracy is limited to about the square root of the machine epsilon. Here, we propose an extension to Riemannian manifolds of the line search of Hager and Zhang, which uses approximate Wolfe conditions that enable computing a solution on the order of the machine epsilon. Numerical experiments demonstrate the computational efficiency of the proposed framework.

Paper Structure

This paper contains 31 sections, 1 theorem, 77 equations, 13 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Preliminaries on multilevel optimization and geometry of fixed-rank matrices
  3. Multilevel optimization in Euclidean space
  4. The manifold of fixed-rank matrices
  5. The orthographic retraction
  6. Riemannian multigrid line search for low-rank matrices
  7. Description of the scheme
  8. Tensor-product multigrid
  9. Riemannian transfer operators
  10. Smoothers
  11. The Riemannian coarse-grid correction
  12. Gradient of the coarse-grid model
  13. Final algorithm: Riemannian multigrid line search
  14. A more accurate line search
  15. Inaccuracy in standard line search
  16. ...and 16 more sections

Key Result

Proposition 2.1

The set ${\mathcal{M}}_{k}$ is a smooth submanifold of dimension $(m+n-k)k$ embedded in $\mathbb{R}^{m \times n}$. Its tangent space $T_{X}{\mathcal{M}}_{k}$ at $X = U\Sigma V^{\mkern-1mu\textsf{T}} \in {\mathcal{M}}_{k}$ is given by In addition, every tangent vector $\xi \in T_{X}{\mathcal{M}}_{k}$ can be written as with $M \in \mathbb{R}^{k\times k}$, $U_{\mathrm{p}} \in \mathbb{R}^{m\times k}

Figures (13)

  • Figure 1: A two-grid cycle for minimizing an objective function.
  • Figure 1: The Riemannian multigrid line-search (RMGLS) scheme. The coarse-grid correction is computed either directly or by a recursive application of RMGLS. It is instructive to compare this figure to the Euclidean version in \ref{['fig:scheme_of_MGOPT']}.
  • Figure 1: Exact and numerical graphs of $f(x) = 1-2x + x^{2}$ near $x=1$ (adapted from Hager:2005). The dotted line is the exact $f$, while the solid line is its representation in double precision with $\varepsilon_{\mathrm{mach}} \approx 10^{-16}$.
  • Figure 1: Convergence of $\mathrm{err}\textrm{-}{\mathcal{F}}$ and $\mathrm{R\textrm{-}grad}$ for level $\ell_{\mathrm{f}} = 8$ and rank $k = 5$, for the problem of \ref{['sec:var_pb_1']}.
  • Figure 2: The orthographic retraction.
  • ...and 8 more figures

Theorems & Definitions (4)

  • Proposition 2.1: Vandereycken:2013
  • Remark 3.1
  • Remark 3.2
  • Remark 3.3