Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The average condition number of most tensor rank decomposition problems is infinite

Carlos Beltrán, Paul Breiding, Nick Vannieuwenhoven

TL;DR

This work analyzes the numerical conditioning of tensor rank decompositions (CPD) by defining the standard CPD condition number κ and the angular condition number κ_{ang}. It proves that the average value of κ is infinite for random input tensors of rank 2 (and, under mild identifiability assumptions, for higher ranks), highlighting severe average ill‑conditioning in CPD computations; in contrast, the angular CN κ_{ang} has finite expectation for rank‑2 and appears finite in practice for higher ranks. The analysis combines a parametric Jacobian approach, the coarea formula, and lower bound arguments for Jacobians to establish divergence, with extensions to higher ranks via a decomposition argument and Jacobian control (Lemma important_lemma). Numerical experiments using Gaussian‑identified tensors and homotopy continuation corroborate the theoretical findings, showing heavy‑tailed CN distributions for κ and finite angular CN. These results have direct implications for algorithm design and testing, suggesting that recovering angular information is more stable on average than recovering full CPD, and that average CPD computation remains inherently ill‑conditioned in high dimensions.

Abstract

The tensor rank decomposition, or canonical polyadic decomposition, is the decomposition of a tensor into a sum of rank-1 tensors. The condition number of the tensor rank decomposition measures the sensitivity of the rank-1 summands with respect to structured perturbations. Those are perturbations preserving the rank of the tensor that is decomposed. On the other hand, the angular condition number measures the perturbations of the rank-1 summands up to scaling. We show for random rank-2 tensors that the expected value of the condition number is infinite for a wide range of choices of the density. Under a mild additional assumption, we show that the same is true for most higher ranks $r\geq 3$ as well. In fact, as the dimensions of the tensor tend to infinity, asymptotically all ranks are covered by our analysis. On the contrary, we show that rank-2 tensors have finite expected angular condition number. Based on numerical experiments, we conjecture that this could also be true for higher ranks. Our results underline the high computational complexity of computing tensor rank decompositions. We discuss consequences of our results for algorithm design and for testing algorithms computing tensor rank decompositions.

The average condition number of most tensor rank decomposition problems is infinite

TL;DR

This work analyzes the numerical conditioning of tensor rank decompositions (CPD) by defining the standard CPD condition number κ and the angular condition number κ_{ang}. It proves that the average value of κ is infinite for random input tensors of rank 2 (and, under mild identifiability assumptions, for higher ranks), highlighting severe average ill‑conditioning in CPD computations; in contrast, the angular CN κ_{ang} has finite expectation for rank‑2 and appears finite in practice for higher ranks. The analysis combines a parametric Jacobian approach, the coarea formula, and lower bound arguments for Jacobians to establish divergence, with extensions to higher ranks via a decomposition argument and Jacobian control (Lemma important_lemma). Numerical experiments using Gaussian‑identified tensors and homotopy continuation corroborate the theoretical findings, showing heavy‑tailed CN distributions for κ and finite angular CN. These results have direct implications for algorithm design and testing, suggesting that recovering angular information is more stable on average than recovering full CPD, and that average CPD computation remains inherently ill‑conditioned in high dimensions.

Abstract

The tensor rank decomposition, or canonical polyadic decomposition, is the decomposition of a tensor into a sum of rank-1 tensors. The condition number of the tensor rank decomposition measures the sensitivity of the rank-1 summands with respect to structured perturbations. Those are perturbations preserving the rank of the tensor that is decomposed. On the other hand, the angular condition number measures the perturbations of the rank-1 summands up to scaling. We show for random rank-2 tensors that the expected value of the condition number is infinite for a wide range of choices of the density. Under a mild additional assumption, we show that the same is true for most higher ranks as well. In fact, as the dimensions of the tensor tend to infinity, asymptotically all ranks are covered by our analysis. On the contrary, we show that rank-2 tensors have finite expected angular condition number. Based on numerical experiments, we conjecture that this could also be true for higher ranks. Our results underline the high computational complexity of computing tensor rank decompositions. We discuss consequences of our results for algorithm design and for testing algorithms computing tensor rank decompositions.

Paper Structure

This paper contains 34 sections, 24 theorems, 227 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. The condition number of tensor rank decomposition
  3. Informal version of our main results and discussion.
  4. Identifiable tensors and a formula for the condition number
  5. Main results
  6. Acknowledgements
  7. Organization of the article
  8. Notation and Preliminaries
  9. Notation
  10. Differential geometry
  11. The manifold of $r$-nice tensors
  12. Sensitivity of CPDs
  13. Integrals
  14. The coarea formula
  15. The average condition number of Gaussian tensors of rank two
  16. ...and 19 more sections

Key Result

Lemma 1.1

Let $\mathbb{F}$ be $\mathbb{R}$ or $\mathbb{C}$, $\mathpzc{A}\in \mathbb{F}^{n_1\times n_2\times n_3}$ a tensor of order $3$ and assume that $\mathpzc{A} = \sum_{i=1}^r \mathpzc{A}_i,$ where $\mathpzc{A}_i = \lambda_i \mathbf{u}_{i}^{1} \otimes \mathbf{u}_{i}^{2}\otimes\mathbf{u}_{i}^{3} \in \mathb

Figures (1)

  • Figure 7.1: Empirical complementary cumulative distribution function of the regular and angular condition numbers for the tensor spaces from \ref{['tab_homo_sols']}. Both plots are on the same scale.

Theorems & Definitions (46)

  • Lemma 1.1: Kruskal's criterion Kruskal1977SGB2000
  • Remark 1.2
  • Definition 1.3: Gaussian Identifiable Tensors
  • Remark 1.4
  • Theorem 1.5
  • Theorem 1.6
  • Corollary 1.7
  • Conjecture 1.8
  • Theorem 1.9
  • Conjecture 1.10
  • ...and 36 more