Lower Bounds on Adaptive Sensing for Matrix Recovery

TL;DR

The paper establishes information-theoretic lower bounds for adaptive sensing in low-rank matrix recovery, showing that even with multiple adaptive rounds, the total measurement budget cannot be reduced below near-reading the entire matrix in many regimes. By embedding a rank-$r$ Gaussian spike into a Gaussian background and leveraging Bayes risk bounds and Gaussian rotational invariance, it proves that the number of rounds $t$ satisfies $t = ilde{ ext{Ω}}ig(rac{ ext{log}(n^2/k)}{ ext{log} ext{log} n}ig)$ for $k$ measurements per round, implying $ ext{Ω}(n^2)$ total measurements when rounds are sublogarithmic. The results extend to tensor recovery and yield a broad rounds-vs-measurements toolbox for numerous numerical linear algebra tasks (spectral/Frobenius/Schatten/Ky-Fan; reduced rank regression; singular vector approximation) under general linear measurements with Gaussian noise. Conceptually, the work shows that adaptivity provides limited leverage over prior non-adaptive bounds in this sensing model, and it connects to existing upper-bound algorithms by demonstrating near-optimal round complexities in many well-conditioned scenarios.

Abstract

We study lower bounds on adaptive sensing algorithms for recovering low rank matrices using linear measurements. Given an $n \times n$ matrix $A$, a general linear measurement $S(A)$, for an $n \times n$ matrix $S$, is just the inner product of $S$ and $A$, each treated as $n^2$-dimensional vectors. By performing as few linear measurements as possible on a rank-$r$ matrix $A$, we hope to construct a matrix $\hat{A}$ that satisfies $\|A - \hat{A}\|_F^2 \le c\|A\|_F^2$, for a small constant $c$. It is commonly assumed that when measuring $A$ with $S$, the response is corrupted with an independent Gaussian random variable of mean $0$ and variance $σ^2$. Candés and Plan study non-adaptive algorithms for low rank matrix recovery using random linear measurements. At a certain noise level, it is known that their non-adaptive algorithms need to perform $Ω(n^2)$ measurements, which amounts to reading the entire matrix. An important question is whether adaptivity helps in decreasing the overall number of measurements. We show that any adaptive algorithm that uses $k$ linear measurements in each round and outputs an approximation to the underlying matrix with probability $\ge 9/10$ must run for $t = Ω(\log(n^2/k)/\log\log n)$ rounds showing that any adaptive algorithm which uses $n^{2-β}$ linear measurements in each round must run for $Ω(\log n/\log\log n)$ rounds to compute a reconstruction with probability $\ge 9/10$. Hence any adaptive algorithm that has $o(\log n/\log\log n)$ rounds must use an overall $Ω(n^2)$ linear measurements. Our techniques also readily extend to obtain lower bounds on adaptive algorithms for tensor recovery and obtain measurement-vs-rounds trade-off for many sensing problems in numerical linear algebra, such as spectral norm low rank approximation, Frobenius norm low rank approximation, singular vector approximation, and more.

Theorems & Definitions (42)

• Theorem 1.1
• Theorem 2.1
• Theorem 2.2
• Lemma 2.3
• proof
• Lemma 2.4: Chain Rule
• Lemma 2.5: KL-divergence between Gaussians, folklore
• Lemma 3.1
• Lemma 3.2
• proof