Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From low-rank retractions to dynamical low-rank approximation and back

Axel Séguin, Gianluca Ceruti, Daniel Kressner

TL;DR

This work bridges retractions from differential geometry with numerical integration for DLRA on fixed-rank matrix manifolds. By showing that existing DLRA schemes correspond to specific low-rank retractions (notably SVD, KSL, and KLS), it provides a geometric interpretation of DLRA methods and introduces two new retraction-based time-stepping schemes: the Accelerated Forward Euler (AFE) and the Projected Ralston--Hermite (PRH) methods, with an accelerated variant APRH. The schemes achieve local truncation error of order three and are analyzed via the Weingarten map to account for manifold curvature, enabling stable integration even for small singular values in some cases. Numerical experiments on the differential Lyapunov equation and small-singular-value tests illustrate the strengths and limitations of each method, highlighting robustness in PRH and sensitivity of AFE/APRH to model errors or curvature effects. The results advance practical, geometry-aware DLRA integration and suggest directions for extending retraction-based techniques to broader low-rank formats.

Abstract

In algorithms for solving optimization problems constrained to a smooth manifold, retractions are a well-established tool to ensure that the iterates stay on the manifold. More recently, it has been demonstrated that retractions are a useful concept for other computational tasks on manifold as well, including interpolation tasks. In this work, we consider the application of retractions to the numerical integration of differential equations on fixed-rank matrix manifolds. This is closely related to dynamical low-rank approximation (DLRA) techniques. In fact, any retraction leads to a numerical integrator and, vice versa, certain DLRA techniques bear a direct relation with retractions. As an example for the latter, we introduce a new retraction, called KLS retraction, that is derived from the so-called unconventional integrator for DLRA. We also illustrate how retractions can be used to recover known DLRA techniques and to design new ones. In particular, this work introduces two novel numerical integration schemes that apply to differential equations on general manifolds: the accelerated forward Euler (AFE) method and the Projected Ralston--Hermite (PRH) method. Both methods build on retractions by using them as a tool for approximating curves on manifolds. The two methods are proven to have local truncation error of order three. Numerical experiments on classical DLRA examples highlight the advantages and shortcomings of these new methods.

From low-rank retractions to dynamical low-rank approximation and back

TL;DR

This work bridges retractions from differential geometry with numerical integration for DLRA on fixed-rank matrix manifolds. By showing that existing DLRA schemes correspond to specific low-rank retractions (notably SVD, KSL, and KLS), it provides a geometric interpretation of DLRA methods and introduces two new retraction-based time-stepping schemes: the Accelerated Forward Euler (AFE) and the Projected Ralston--Hermite (PRH) methods, with an accelerated variant APRH. The schemes achieve local truncation error of order three and are analyzed via the Weingarten map to account for manifold curvature, enabling stable integration even for small singular values in some cases. Numerical experiments on the differential Lyapunov equation and small-singular-value tests illustrate the strengths and limitations of each method, highlighting robustness in PRH and sensitivity of AFE/APRH to model errors or curvature effects. The results advance practical, geometry-aware DLRA integration and suggest directions for extending retraction-based techniques to broader low-rank formats.

Abstract

In algorithms for solving optimization problems constrained to a smooth manifold, retractions are a well-established tool to ensure that the iterates stay on the manifold. More recently, it has been demonstrated that retractions are a useful concept for other computational tasks on manifold as well, including interpolation tasks. In this work, we consider the application of retractions to the numerical integration of differential equations on fixed-rank matrix manifolds. This is closely related to dynamical low-rank approximation (DLRA) techniques. In fact, any retraction leads to a numerical integrator and, vice versa, certain DLRA techniques bear a direct relation with retractions. As an example for the latter, we introduce a new retraction, called KLS retraction, that is derived from the so-called unconventional integrator for DLRA. We also illustrate how retractions can be used to recover known DLRA techniques and to design new ones. In particular, this work introduces two novel numerical integration schemes that apply to differential equations on general manifolds: the accelerated forward Euler (AFE) method and the Projected Ralston--Hermite (PRH) method. Both methods build on retractions by using them as a tool for approximating curves on manifolds. The two methods are proven to have local truncation error of order three. Numerical experiments on classical DLRA examples highlight the advantages and shortcomings of these new methods.
Paper Structure (27 sections, 4 theorems, 58 equations, 11 figures, 2 tables, 4 algorithms)

This paper contains 27 sections, 4 theorems, 58 equations, 11 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Outline
  3. Preliminaries
  4. Differential geometry
  5. Manifold of rank-$r$ matrices.
  6. Retractions.
  7. DLRA
  8. DLRA algorithms and low-rank retractions
  9. Metric-projection retraction
  10. Projector-splitting KSL retraction
  11. Projector-splitting KLS retraction
  12. The KLS retraction and the orthographic retraction
  13. The orthographic retraction.
  14. New retraction-based time stepping algorithms
  15. An accelerated forward Euler scheme
  16. ...and 12 more sections

Key Result

Proposition 1

The procedure of Algorithm alg:retractionKLS defines a second-order retraction (called the KLS retraction).

Figures (11)

  • Figure 1: Orthographic retraction and its inverse.
  • Figure 2: $\left\lVert Q\right\rVert_{\mathrm{F}} = 0$.
  • Figure 3: $\left\lVert Q\right\rVert_{\mathrm{F}} = 0.01$.
  • Figure 4: $\left\lVert Q\right\rVert_{\mathrm{F}} = 0.1$.
  • Figure 5: $\left\lVert Q\right\rVert_{\mathrm{F}} = 1$.
  • ...and 6 more figures

Theorems & Definitions (8)

  • Proposition 1
  • proof
  • Proposition 2: boumalBook, myThesis
  • Definition 3
  • Definition 4
  • Proposition 5: extrinsiclook
  • Proposition 6
  • proof