Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

First-order methods on bounded-rank tensors converging to stationary points

Bin Gao, Renfeng Peng, Ya-xiang Yuan

TL;DR

This paper tackles the challenge of provably finding stationary points of smooth objectives over the non-smooth set of bounded Tucker-rank tensors $\mathcal{M}_{\leq\mathbf{r}}$. It develops two first-order methods, GRAP-R and rfGRAP-R, grounded in an explicit normal-cone description and gradient-related approximate projections, augmented by a rank-decrease mechanism to handle rank-deficient points. The authors prove that all accumulation points of the generated sequences are stationary, establishing apocalypse-free convergence under standard smoothness assumptions, and validate the methods on tensor-completion tasks. The results advance low-rank tensor optimization by delivering practical, provably convergent algorithms with controlled computational cost and robustness to rank mis-specification, offering a tractable alternative to desingularization or lifted-problem approaches.

Abstract

Provably finding stationary points on bounded-rank tensors turns out to be an open problem [E. Levin, J. Kileel, and N. Boumal, Math. Program., 199 (2023), pp. 831--864] due to the inherent non-smoothness of the set of bounded-rank tensors. We resolve this problem by proposing two first-order methods with guaranteed convergence to stationary points. Specifically, we revisit the variational geometry of bounded-rank tensors and explicitly characterize its normal cones. Moreover, we propose gradient-related approximate projection methods that are provable to find stationary points, where the decisive ingredients are gradient-related vectors from tangent cones, line search along approximate projections, and rank-decreasing mechanisms near rank-deficient points. Numerical experiments on tensor completion validate that the proposed methods converge to stationary points across various rank parameters.

First-order methods on bounded-rank tensors converging to stationary points

TL;DR

This paper tackles the challenge of provably finding stationary points of smooth objectives over the non-smooth set of bounded Tucker-rank tensors . It develops two first-order methods, GRAP-R and rfGRAP-R, grounded in an explicit normal-cone description and gradient-related approximate projections, augmented by a rank-decrease mechanism to handle rank-deficient points. The authors prove that all accumulation points of the generated sequences are stationary, establishing apocalypse-free convergence under standard smoothness assumptions, and validate the methods on tensor-completion tasks. The results advance low-rank tensor optimization by delivering practical, provably convergent algorithms with controlled computational cost and robustness to rank mis-specification, offering a tractable alternative to desingularization or lifted-problem approaches.

Abstract

Provably finding stationary points on bounded-rank tensors turns out to be an open problem [E. Levin, J. Kileel, and N. Boumal, Math. Program., 199 (2023), pp. 831--864] due to the inherent non-smoothness of the set of bounded-rank tensors. We resolve this problem by proposing two first-order methods with guaranteed convergence to stationary points. Specifically, we revisit the variational geometry of bounded-rank tensors and explicitly characterize its normal cones. Moreover, we propose gradient-related approximate projection methods that are provable to find stationary points, where the decisive ingredients are gradient-related vectors from tangent cones, line search along approximate projections, and rank-decreasing mechanisms near rank-deficient points. Numerical experiments on tensor completion validate that the proposed methods converge to stationary points across various rank parameters.

Paper Structure

This paper contains 24 sections, 13 theorems, 93 equations, 8 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related work and motivation
  3. Contribution
  4. Organization
  5. Geometry of Tucker tensor varieties
  6. Notations for matrix computations
  7. Geometry of tensors in Tucker decomposition
  8. Projection onto $\mathcal{M}_{\leq\mathbf{r}}$
  9. Tucker tensor varieties
  10. Normal cones and approximate projections
  11. Normal cones and optimality conditions
  12. Approximate projections
  13. Gradient-related approximate projection method with rank decrease (GRAP-R)
  14. Algorithm
  15. Backtracking line search
  16. ...and 9 more sections

Key Result

Proposition 2.1

\newlabelprop: rank delta0 Given $\underline{\mathbf{X}}\in\mathbb{R}^{m\times n}_{\underline{r}}$ and $\Delta>0$. It holds that $\mathrm{rank}_\Delta\mathbf{X}\leq\underline{r}$ for all $\mathbf{X}\in B[\underline{\mathbf{X}},\Delta]$. If $\underline{\mathbf{X}}\neq 0$, it holds that $\underline{

Figures (8)

  • Figure 1: A general framework of proposed first-order methods. The surfaces represent sets of fixed-rank tensors, and $\tilde{\mathcal{V}}^{(t)}$ is a search direction. Green points are generated from lower-rank candidates via line search.
  • Figure 1: Tucker decomposition of a third-order tensor.
  • Figure 1: Illustration of a vector in normal cone for $d=3$. $G_k^\perp=\{\tilde{\mathcal{G}}:\tilde{\mathbf{G}}_{(k)}\in\mathbb{R}^{(n_k-\underline{r}_k)\times r_{-k}},\tilde{\mathbf{G}}_{(k)}^{}\mathbf{G}_{(k)}^\top=0\}$ for $k\in[3]$. Left: $I=\{2,3\}$. Middle: $I=\{1\}$. Right: $I=\emptyset$.
  • Figure 1: Proof sketch of \ref{['prop: GRAP-R decrease']}.
  • Figure 1: Proof sketch of \ref{['prop: rfGRAP']}.
  • ...and 3 more figures

Theorems & Definitions (25)

  • Proposition 2.1: olikier2024low
  • Definition 2.2: Tucker decomposition
  • Proposition 3.1
  • Lemma 3.2
  • Proof 1
  • Proposition 3.3
  • Proof 2
  • Proposition 3.4
  • Proof 3
  • Lemma 4.1
  • ...and 15 more