Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An apocalypse-free first-order low-rank optimization algorithm with at most one rank reduction attempt per iteration

Guillaume Olikier, P. -A. Absil

TL;DR

The paper addresses minimizing a differentiable function with locally Lipschitz gradient over the determinantal variety $\mathbb{R}_{\le r}^{m\times n}$ by introducing the apocalypse-free RFDR algorithm. RFDR combines the retraction-free descent of $RFD$ with a rank-reduction check that reduces rank by at most one per iteration, guaranteeing that all accumulation points are stationary and avoiding apocalyptic behavior. Compared to $P^2$GDR, RFDR offers lower computational overhead by limiting rank-reduction attempts while preserving the apocalypse-free property and staying retraction-free. The authors provide a convergence proof within the Polak framework and a practical cost analysis highlighting RFDR's competitive efficiency, while also discussing open questions about removing the need for large-scale SVDs entirely.

Abstract

We consider the problem of minimizing a differentiable function with locally Lipschitz continuous gradient over the real determinantal variety, and present a first-order algorithm designed to find stationary points of that problem. This algorithm applies steps of a retraction-free descent method proposed by Schneider and Uschmajew (2015), while taking the numerical rank into account to attempt rank reductions. We prove that this algorithm produces a sequence of iterates the accumulation points of which are stationary, and therefore does not follow the so-called apocalypses described by Levin, Kileel, and Boumal (2022). Moreover, the rank reduction mechanism of this algorithm requires at most one rank reduction attempt per iteration, in contrast with the one of the $\mathrm{P}^2\mathrm{GDR}$ algorithm introduced by Olikier, Gallivan, and Absil (2022) which can require a number of rank reduction attempts equal to the rank of the iterate in the worst-case scenario.

An apocalypse-free first-order low-rank optimization algorithm with at most one rank reduction attempt per iteration

TL;DR

The paper addresses minimizing a differentiable function with locally Lipschitz gradient over the determinantal variety by introducing the apocalypse-free RFDR algorithm. RFDR combines the retraction-free descent of with a rank-reduction check that reduces rank by at most one per iteration, guaranteeing that all accumulation points are stationary and avoiding apocalyptic behavior. Compared to GDR, RFDR offers lower computational overhead by limiting rank-reduction attempts while preserving the apocalypse-free property and staying retraction-free. The authors provide a convergence proof within the Polak framework and a practical cost analysis highlighting RFDR's competitive efficiency, while also discussing open questions about removing the need for large-scale SVDs entirely.

Abstract

We consider the problem of minimizing a differentiable function with locally Lipschitz continuous gradient over the real determinantal variety, and present a first-order algorithm designed to find stationary points of that problem. This algorithm applies steps of a retraction-free descent method proposed by Schneider and Uschmajew (2015), while taking the numerical rank into account to attempt rank reductions. We prove that this algorithm produces a sequence of iterates the accumulation points of which are stationary, and therefore does not follow the so-called apocalypses described by Levin, Kileel, and Boumal (2022). Moreover, the rank reduction mechanism of this algorithm requires at most one rank reduction attempt per iteration, in contrast with the one of the algorithm introduced by Olikier, Gallivan, and Absil (2022) which can require a number of rank reduction attempts equal to the rank of the iterate in the worst-case scenario.
Paper Structure (8 sections, 6 theorems, 42 equations, 1 table, 8 algorithms)

This paper contains 8 sections, 6 theorems, 42 equations, 1 table, 8 algorithms.

Table of Contents

  1. Introduction
  2. Notation
  3. The restricted tangent cone to the determinantal variety
  4. The $\mathrm{RFD}$ map
  5. The proposed algorithm
  6. Convergence analysis
  7. Practical implementation and computational cost
  8. Conclusion

Key Result

Proposition 3.2

\newlabelprop:ProjectionRestrictedTangentConeDeterminantalVariety0 Let $Z \in \mathbb{R}^{m \times n}$ be written as with $A = U^\top Z V$, $B = U^\top Z V_\perp$, $C = U_\perp^\top Z V$, and $D = U_\perp^\top Z V_\perp$. Then, Moreover, and, for all $\breve{Z} \in P_{{ \mathop{\hbox{\m@th\ialign{\crcr $\m@th \hbox{origin=c]{90}{\upshape(}}$\crcr{} $T$\crcr}}}\limits}_{\mathbb{R}_{\le r}^

Theorems & Definitions (12)

  • Definition 3.1
  • Proposition 3.2
  • Proof 1
  • Proposition 4.1
  • Proof 2
  • Corollary 4.2
  • Proof 3
  • Proposition 6.1
  • Proof 4
  • Theorem 6.2
  • ...and 2 more