Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Information-Theoretic Guarantees for Recovering Low-Rank Tensors from Symmetric Rank-One Measurements

Eren C. Kızıldağ

TL;DR

This work derives information-theoretic guarantees for exactly recovering a low symmetric-rank order-$\ell$ tensor from symmetric rank-one measurements with i.i.d. log-concave entries. The authors prove a near-optimal sample complexity of $N=\Theta(dr)$ for exact recovery via a rank-minimization program, and complement it with a Fano-based lower bound showing that $N=\tilde{\Omega}(dr^{0.98})$ is necessary up to polylog factors, underscoring the efficiency of the low-rank approach. They connect these results to learning two-layer polynomial networks, demonstrate that ERM over unstructured tensors can require exponentially more samples, and develop a suite of tools—covering numbers, Carbery–Wright anti-concentration, and orthogonal-polynomial expansions—to handle symmetric rank-one measurements under log-concave distributions. The paper highlights the potential for information-theoretic benchmarks to guide the design of tractable algorithms and deepens understanding of sample complexity in structured tensor recovery and polynomial-network settings.

Abstract

In this paper, we investigate the sample complexity of recovering tensors with low symmetric rank from symmetric rank-one measurements. This setting is particularly motivated by the study of higher-order interactions and the analysis of two-layer neural networks with polynomial activations (polynomial networks). Using a covering numbers argument, we analyze the performance of the symmetric rank minimization program and establish near-optimal sample complexity bounds when the underlying distribution is log-concave. Our measurement model involves random symmetric rank-one tensors, which lead to involved probability calculations. To address these challenges, we employ the Carbery-Wright inequality, a powerful tool for studying anti-concentration properties of random polynomials, and leverage orthogonal polynomials. Additionally, we provide a sample complexity lower bound based on Fano's inequality, and discuss broader implications of our results for two-layer polynomial networks.

Information-Theoretic Guarantees for Recovering Low-Rank Tensors from Symmetric Rank-One Measurements

TL;DR

This work derives information-theoretic guarantees for exactly recovering a low symmetric-rank order- tensor from symmetric rank-one measurements with i.i.d. log-concave entries. The authors prove a near-optimal sample complexity of for exact recovery via a rank-minimization program, and complement it with a Fano-based lower bound showing that is necessary up to polylog factors, underscoring the efficiency of the low-rank approach. They connect these results to learning two-layer polynomial networks, demonstrate that ERM over unstructured tensors can require exponentially more samples, and develop a suite of tools—covering numbers, Carbery–Wright anti-concentration, and orthogonal-polynomial expansions—to handle symmetric rank-one measurements under log-concave distributions. The paper highlights the potential for information-theoretic benchmarks to guide the design of tractable algorithms and deepens understanding of sample complexity in structured tensor recovery and polynomial-network settings.

Abstract

In this paper, we investigate the sample complexity of recovering tensors with low symmetric rank from symmetric rank-one measurements. This setting is particularly motivated by the study of higher-order interactions and the analysis of two-layer neural networks with polynomial activations (polynomial networks). Using a covering numbers argument, we analyze the performance of the symmetric rank minimization program and establish near-optimal sample complexity bounds when the underlying distribution is log-concave. Our measurement model involves random symmetric rank-one tensors, which lead to involved probability calculations. To address these challenges, we employ the Carbery-Wright inequality, a powerful tool for studying anti-concentration properties of random polynomials, and leverage orthogonal polynomials. Additionally, we provide a sample complexity lower bound based on Fano's inequality, and discuss broader implications of our results for two-layer polynomial networks.

Paper Structure

This paper contains 30 sections, 16 theorems, 90 equations.

Table of Contents

  1. Introduction
  2. Symmetric Rank-One Measurements
  3. Distributional Assumption
  4. Low Symmetric Rank Tensors
  5. Motivation
  6. Connections to Learning Two-Layer Polynomial Networks
  7. Notation
  8. Main Results
  9. Implications for Neural Networks
  10. Comparison with Empirical Risk Minimization
  11. Sample Complexity Lower Bounds
  12. Comparison with the Prior Work on Low-Rank Tensor/Matrix Recovery
  13. Comparison with Sample Complexity Bounds for Neural Networks
  14. Information-Theoretic Bounds and Noiseless Models
  15. Information-Theoretic Results
  16. ...and 15 more sections

Key Result

Theorem 2.1

Let $C>2\ell^2$ be an arbitrary constant and $N\ge Crd$. Then, eq:Recovery-Program recovers all $\boldsymbol{\mathcal{T}}^*$ with $\mathrm{rank}_S(\boldsymbol{\mathcal{T}}^*)\le r$ with probability one.

Theorems & Definitions (27)

  • Remark 1.1
  • Theorem 2.1
  • Theorem 2.2
  • Theorem 2.3
  • Proposition 2.4
  • Theorem 2.5
  • Proposition 3.1
  • Definition 4.1
  • Definition 4.2
  • Lemma 4.3
  • ...and 17 more