Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Guaranteed Noisy CP Tensor Recovery via Riemannian Optimization on the Segre Manifold

Ke Xu, Yuefeng Han

TL;DR

This work tackles the challenge of recovering a low-CP-rank tensor from noisy linear measurements by optimizing directly on the Segre manifold, the space of rank-one tensors. It introduces two Riemannian optimization algorithms, RGD and RGN, that preserve feasibility at every iteration and leverage the manifold geometry to achieve favorable local convergence: linear for RGD and a two-phase, quadratic-to-linear convergence for RGN, up to a noise floor. Theoretical results establish contraction bounds under mild incoherence and standard initialization, with detailed computational- and sample-complexity analyses. Empirical evaluations on tensor decomposition and tensor regression demonstrate competitive or superior performance relative to CP-ALS, ICO, and RRR, and highlight robustness to noise and non-orthogonal factors. The framework offers a unifying, provably convergent approach for CP-tensor estimation with potential extensions to completion and streaming scenarios, providing both practical algorithms and rigorous guarantees for high-dimensional, noisy tensor data.

Abstract

Recovering a low-CP-rank tensor from noisy linear measurements is a central challenge in high-dimensional data analysis, with applications spanning tensor PCA, tensor regression, and beyond. We exploit the intrinsic geometry of rank-one tensors by casting the recovery task as an optimization problem over the Segre manifold, the smooth Riemannian manifold of rank-one tensors. This geometric viewpoint yields two powerful algorithms: Riemannian Gradient Descent (RGD) and Riemannian Gauss-Newton (RGN), each of which preserves feasibility at every iteration. Under mild noise assumptions, we prove that RGD converges at a local linear rate, while RGN exhibits an initial local quadratic convergence phase that transitions to a linear rate as the iterates approach the statistical noise floor. Extensive synthetic experiments validate these convergence guarantees and demonstrate the practical effectiveness of our methods.

Guaranteed Noisy CP Tensor Recovery via Riemannian Optimization on the Segre Manifold

TL;DR

This work tackles the challenge of recovering a low-CP-rank tensor from noisy linear measurements by optimizing directly on the Segre manifold, the space of rank-one tensors. It introduces two Riemannian optimization algorithms, RGD and RGN, that preserve feasibility at every iteration and leverage the manifold geometry to achieve favorable local convergence: linear for RGD and a two-phase, quadratic-to-linear convergence for RGN, up to a noise floor. Theoretical results establish contraction bounds under mild incoherence and standard initialization, with detailed computational- and sample-complexity analyses. Empirical evaluations on tensor decomposition and tensor regression demonstrate competitive or superior performance relative to CP-ALS, ICO, and RRR, and highlight robustness to noise and non-orthogonal factors. The framework offers a unifying, provably convergent approach for CP-tensor estimation with potential extensions to completion and streaming scenarios, providing both practical algorithms and rigorous guarantees for high-dimensional, noisy tensor data.

Abstract

Recovering a low-CP-rank tensor from noisy linear measurements is a central challenge in high-dimensional data analysis, with applications spanning tensor PCA, tensor regression, and beyond. We exploit the intrinsic geometry of rank-one tensors by casting the recovery task as an optimization problem over the Segre manifold, the smooth Riemannian manifold of rank-one tensors. This geometric viewpoint yields two powerful algorithms: Riemannian Gradient Descent (RGD) and Riemannian Gauss-Newton (RGN), each of which preserves feasibility at every iteration. Under mild noise assumptions, we prove that RGD converges at a local linear rate, while RGN exhibits an initial local quadratic convergence phase that transitions to a linear rate as the iterates approach the statistical noise floor. Extensive synthetic experiments validate these convergence guarantees and demonstrate the practical effectiveness of our methods.

Paper Structure

This paper contains 44 sections, 9 theorems, 86 equations, 7 figures, 1 table, 8 algorithms.

Table of Contents

  1. Introduction
  2. Main contribution.
  3. Related Work
  4. Organization
  5. Model and Problem Formulation
  6. Tensor Decomposition.
  7. Tensor Regression.
  8. Method
  9. Background and Preliminaries
  10. Riemann Gradient Descent on Segre Manifold
  11. Tangent Space of the Segre Manifold.
  12. Retraction
  13. Riemann Gauss-Newton on Segre Manifold
  14. Theoretical Analysis
  15. Convergence Analysis of Riemann Optimization
  16. ...and 29 more sections

Key Result

Theorem 4.1

Suppose that for each $i \in [r]$, the current estimate $\mathcal{T}_i^{(t)}$ at iteration $t$ satisfies $\langle \mathcal{T}_i^{(t)}, \mathcal{T}_i\rangle \geq 0$, where $\mathcal{T}_i$ is the true rank-one tensor. Define $\varepsilon^{(t)} = \max_{i \in [r]} (\|\mathcal{T}_i^{(t)}-\mathcal{T}_i\|_

Figures (7)

  • Figure 1: Convergence of RGD and RGN for (a) CP decomposition and (b) tensor regression, plotted in terms of relative Frobenius error versus iteration.
  • Figure 2: Convergence of CP tensor decomposition algorithms in terms of relative Frobenius error versus iteration: RGD-SM and RGN-SM (proposed) compared with CP-ALS kolda2009tensor and ICO han2022tensor.
  • Figure 3: Convergence of CP tensor regression algorithms in terms of relative Frobenius error versus iteration: RGD-SM and RGN-SM (proposed) compared with CP-ALS kolda2009tensor and RRR lock2018tensor.
  • Figure 4: Convergence of the relative Frobenius reconstruction error over 30 iterations for various noise scales and coherence numbers. Curves are averaged over all 20 independent replicates.
  • Figure 5: Error distributions after 30 iterations for various noise scales and coherence numbers. Boxes summarize the spread over 20 replicates.
  • ...and 2 more figures

Theorems & Definitions (19)

  • Definition 1: Segre Manifold
  • Remark 1
  • Theorem 4.1: Local Convergence of RGD
  • Theorem 4.2: Local Convergence of RGN
  • Corollary 4.1: Convergence rate of RGD for Tensor CP decomposition
  • Corollary 4.2: Convergence rate of RGN for Tensor CP decomposition
  • Remark 2
  • Corollary 4.3: Convergence rate of RGD for CP tensor regression
  • Corollary 4.4: Convergence rate of RGN for CP tensor regression
  • Remark 3
  • ...and 9 more