Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Riemannian rank-adaptive method for higher-order tensor completion in the tensor-train format

Charlotte Vermeylen, Marc Van Barel

TL;DR

This work addresses low-rank tensor completion in the tensor-train (TT) format by formulating the problem as a least-squares optimization on the TT-variety with bounded ranks. It introduces a Riemannian rank-adaptive method (RRAM) that alternates fixed-rank Riemannian optimization with rank updates guided by descent directions in the tangent cone, plus a rank-reduction mechanism via TT-rounding. A key contribution is a rank-estimation procedure that leverages auxiliary projections to determine how much to increase TT-ranks, along with an angle-conditions-based justification for approximate projections. The method demonstrates faster convergence and reliable rank recovery, outperforming the state-of-the-art in computation time on synthetic data and function-interpolation tasks, with code publicly available.

Abstract

In this paper a new Riemannian rank adaptive method (RRAM) is proposed for the low-rank tensor completion problem (LRTCP) formulated as a least-squares optimization problem on the algebraic variety of tensors of bounded tensor-train (TT) rank. The method iteratively optimizes over fixed-rank smooth manifolds using a Riemannian conjugate gradient algorithm from Steinlechner (2016) and gradually increases the rank by computing a descent direction in the tangent cone to the variety. Additionally, a numerical method to estimate the amount of rank increase is proposed based on a theoretical result for the stationary points of the low-rank tensor approximation problem and a definition of an estimated TT-rank. Furthermore, when the iterate comes close to a lower-rank set, the RRAM decreases the rank based on the TT-rounding algorithm from Oseledets (2011) and a definition of a numerical rank. We prove that the TT-rounding algorithm can be considered as an approximate projection onto the lower-rank set which satisfies a certain angle condition to ensure that the image is sufficiently close to that of an exact projection. Several numerical experiments are given to illustrate the use of the RRAM and its subroutines in {\Matlab}. Furthermore, in all experiments the proposed RRAM outperforms the state-of-the-art RRAM for tensor completion in the TT format from Steinlechner (2016) in terms of computation time.

A Riemannian rank-adaptive method for higher-order tensor completion in the tensor-train format

TL;DR

This work addresses low-rank tensor completion in the tensor-train (TT) format by formulating the problem as a least-squares optimization on the TT-variety with bounded ranks. It introduces a Riemannian rank-adaptive method (RRAM) that alternates fixed-rank Riemannian optimization with rank updates guided by descent directions in the tangent cone, plus a rank-reduction mechanism via TT-rounding. A key contribution is a rank-estimation procedure that leverages auxiliary projections to determine how much to increase TT-ranks, along with an angle-conditions-based justification for approximate projections. The method demonstrates faster convergence and reliable rank recovery, outperforming the state-of-the-art in computation time on synthetic data and function-interpolation tasks, with code publicly available.

Abstract

In this paper a new Riemannian rank adaptive method (RRAM) is proposed for the low-rank tensor completion problem (LRTCP) formulated as a least-squares optimization problem on the algebraic variety of tensors of bounded tensor-train (TT) rank. The method iteratively optimizes over fixed-rank smooth manifolds using a Riemannian conjugate gradient algorithm from Steinlechner (2016) and gradually increases the rank by computing a descent direction in the tangent cone to the variety. Additionally, a numerical method to estimate the amount of rank increase is proposed based on a theoretical result for the stationary points of the low-rank tensor approximation problem and a definition of an estimated TT-rank. Furthermore, when the iterate comes close to a lower-rank set, the RRAM decreases the rank based on the TT-rounding algorithm from Oseledets (2011) and a definition of a numerical rank. We prove that the TT-rounding algorithm can be considered as an approximate projection onto the lower-rank set which satisfies a certain angle condition to ensure that the image is sufficiently close to that of an exact projection. Several numerical experiments are given to illustrate the use of the RRAM and its subroutines in {\Matlab}. Furthermore, in all experiments the proposed RRAM outperforms the state-of-the-art RRAM for tensor completion in the TT format from Steinlechner (2016) in terms of computation time.
Paper Structure (25 sections, 6 theorems, 101 equations, 12 figures, 4 algorithms)

This paper contains 25 sections, 6 theorems, 101 equations, 12 figures, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notation
  4. Properties tensor-train decomposition
  5. Compact notations
  6. Tangent cone
  7. Low-rank tensor approximation problem
  8. Approximate projection
  9. Methodology
  10. Rank increase
  11. Search direction in the tangent cone
  12. Rank estimation
  13. Experiments
  14. Algorithm for rank increase
  15. Rank reduction
  16. ...and 10 more sections

Key Result

Lemma 2.1

Let $X \in \mathbb{R}_{(r_1, \dots, r_{d-1})}^{n_1 \times \cdots \times n_d}$ as in eq:i-orth. Then, $T_X \mathbb{R}_{\le (k_1, \dots, k_{d-1})}^{n_1 \times \cdots \times n_d}$ is the set of all tensors $G \in \mathbb{R}^{n_1 \times \cdots \times n_d}$ that can be decomposed as where $U_i \in \mathbb{R}^{r_{i-1} \times n_i \times s_i}$, $s_i := k_i-r_i$, for $i=1,\dots,{d-1}$, $W_i \in \mathbb{

Figures (12)

  • Figure 1: The first 7 singular values of the matrices in \ref{['eq:Pi']}, the relative gap between these singular values, the singular values of the unfoldings of $A_\Omega$ and $A$, where $X^*$ is obtained after 87 iterations of the CG algorithm, and with $d:=4$, $n_i:=15$, for $i=1,\dots,d$, $r'_i:=3$ and $r_i=1$, for $i=1,\dots,d-1$, and $\rho_{\Omega}:= 0.3$. The norm of the Riemannian gradient that is obtained at $X^*$ is approximately $10^{-11}$.
  • Figure 2: The first 7 singular values of the matrices in \ref{['eq:Pi']}, the relative gap between these singular values, the singular values of the unfoldings of $A_\Omega$ and $A$, where $X^*$ is obtained after 15 iterations of the CG algorithm, and with $d:=4$, $n_i:=15$, for $i=1,\dots,d$, $r'_i:=3$, $r_i:=1$, for $i=1,\dots,d-1$, and $\rho_{\Omega}:= 0.3$. The norm of the Riemannian gradient that is obtained at $X^*$ is approximately $31$.
  • Figure 3: The first 7 singular values of the matrices in \ref{['eq:Pi']}, the relative gap between these singular values, the singular values of the unfoldings of $A_\Omega$, and $A$, where $X^*$ is obtained after 15 iterations of the CG algorithm, and with $d:=4$, $n_i:=15$, for $i=1,\dots,d$, $r':=[2,5,3]$, $r_i:=1$, for $i=1,\dots,d-1$, and $\rho_{\Omega}:= 0.3$. The norm of the Riemannian gradient that is obtained at $X^*$ is approximately $13$.
  • Figure 4: The first 7 singular values of the matrices in \ref{['eq:Pi']}, the relative gap between these singular values, the singular values of the unfoldings of $A_\Omega$, and $A$, where $X$ is obtained after 15 iterations of the CG algorithm, and with $d:=5$, $n_i:=10$, for $i=1,\dots,d$, $r'_i:=3$ and $r_i:=1$, for $i=1,\dots,d-1$, and $\rho_{\Omega}:= 0.3$. The norm of the Riemannian gradient that is obtained at $X^*$ is approximately $12$.
  • Figure 5: The first 7 singular values of the matrices in \ref{['eq:Pi']}, the relative gap between these singular values, the singular values of the unfoldings of $A_\Omega$, and $A$, where noise is added to the data as in \ref{['eq:A_noise']} of size $10^{-1}$, and $X$ is obtained after 15 iterations of the CG algorithm, and with $d:=5$, $n_i:=10$, for $i=1,\dots,d$, $r'_i:=3$ and $r_i:=1$, for $i=1,\dots,d-1$, and $\rho_{\Omega}:= 0.3$. The norm of the Riemannian gradient that is obtained at $X^*$ is approximately $77$.
  • ...and 7 more figures

Theorems & Definitions (15)

  • Lemma 2.1
  • proof
  • Proposition 3.1
  • proof
  • Theorem 3.2
  • proof
  • Proposition 3.3
  • proof
  • Definition 3.4: Estimated rank
  • Theorem 3.5: Angle condition \ref{['alg:TT_rounding']}
  • ...and 5 more