Concentration inequalities for random tensors
Roman Vershynin
TL;DR
This work develops two tensor concentration frameworks for simple random tensors $X = x_1 \otimes \cdots \otimes x_d$ with independent coordinates: a convex concentration theorem for convex Lipschitz functionals and a Euclidean concentration theorem for Euclidean functionals $f(x)=\|Ax\|_H$. Both results achieve optimal, dimension-free dependence on the degree $d$, with precise ranges for the deviation parameter $t$, and rely on a novel maximal inequality controlling products of norms of independent vectors. The authors further apply these results to show that random tensors are well conditioned when the degree satisfies $d = o(\sqrt{n/\log n})$, yielding high-probability lower bounds on linear combinations of tensor samples. The techniques blend telescoping expansions, martingale concentration, MGFs for quadratic forms, and a careful tensorization that avoids exponential loss in $d$, advancing the understanding of tensor concentration beyond prior polynomial-chaos bounds. These results have potential implications for tensor decompositions and higher-order random structures in theoretical computer science and high-dimensional probability.
Abstract
We show how to extend several basic concentration inequalities for simple random tensors $X = x_1 \otimes \cdots \otimes x_d$ where all $x_k$ are independent random vectors in $\mathbb{R}^n$ with independent coefficients. The new results have optimal dependence on the dimension $n$ and the degree $d$. As an application, we show that random tensors are well conditioned: $(1-o(1)) n^d$ independent copies of the simple random tensor $X \in \mathbb{R}^{n^d}$ are far from being linearly dependent with high probability. We prove this fact for any degree $d = o(\sqrt{n/\log n})$ and conjecture that it is true for any $d = O(n)$.