Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Geometric Perspective on the Injective Norm of Sums of Random Tensors

Afonso S. Bandeira, Sivakanth Gopi, Haotian Jiang, Kevin Lucca, Thomas Rothvoss

Abstract

Matrix concentration inequalities, intimately connected to the Non-Commutative Khintchine inequality, have been an important tool in both applied and pure mathematics. We study tensor versions of these inequalities, and establish non-asymptotic inequalities for the $\ell_p$ injective norm of random tensors with correlated entries. In certain regimes of $p$ and the tensor order, our tensor concentration inequalities are nearly optimal in their dimension dependencies. We illustrate our result with applications to problems including structured models of random tensors and matrices, tensor PCA, and connections to lower bounds in coding theory. Our techniques are based on covering number estimates as opposed to operator theoretic tools, which also provide a geometric proof of a weaker version of the Non-Commutative Khintchine inequality, motivated by a question of Talagrand.

A Geometric Perspective on the Injective Norm of Sums of Random Tensors

Abstract

Matrix concentration inequalities, intimately connected to the Non-Commutative Khintchine inequality, have been an important tool in both applied and pure mathematics. We study tensor versions of these inequalities, and establish non-asymptotic inequalities for the injective norm of random tensors with correlated entries. In certain regimes of and the tensor order, our tensor concentration inequalities are nearly optimal in their dimension dependencies. We illustrate our result with applications to problems including structured models of random tensors and matrices, tensor PCA, and connections to lower bounds in coding theory. Our techniques are based on covering number estimates as opposed to operator theoretic tools, which also provide a geometric proof of a weaker version of the Non-Commutative Khintchine inequality, motivated by a question of Talagrand.

Paper Structure

This paper contains 27 sections, 32 theorems, 237 equations.

Table of Contents

  1. Introduction
  2. Main Results
  3. Notation and Terminology
  4. Overview of Techniques
  5. Warm-Up: A Crude Non-Commutative Khintchine Inequality
  6. Applications of Our Results
  7. The Independent Entry Model and Tensor PCA
  8. Matching Matrices
  9. Preliminaries
  10. Variance Parameters and Tensor Norm Inequalities
  11. The Gaussian Process Setting
  12. Classical Covering Number Bounds
  13. Proof of Main Results
  14. Probabilistic Distance Estimates and Diagonal-Free Tensors
  15. Covering Number Bound
  16. ...and 12 more sections

Key Result

Theorem 1.2

Let $r\geq 2$ an integer, $2 \leq p < \infty$, and $\mathcal{C}_{r,p}(d)$ the Type-2 constant of the $\ell_p$ injective norm of order $r$ tensors (see Definition def:type2constant) then (The expression $f \lesssim_{r,p} g$ says that the inequality $f \leq C_{r,p} g$ holds for some constant $C_{r,p}$ that only depends on $r$ and $p$.)

Theorems & Definitions (82)

  • Definition 1.1: Type-2 constant for $\ell_p$ injective norm of order $r$ tensors
  • Theorem 1.2
  • Remark 1.3
  • Definition 1.4: $\star_q$ product
  • Definition 1.5: Variance Parameters
  • Theorem 1.6: Main Theorem
  • Theorem 2.1: Nonhomogeneous independent entry model
  • Corollary 2.2
  • Theorem 2.3: Censored tensor PCA
  • proof : Proof of \ref{['thm:indepentryapplication']}
  • ...and 72 more