Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Computational and statistical lower bounds for low-rank estimation under general inhomogeneous noise

Debsurya De, Dmitriy Kunisky

TL;DR

This work extends the theory of spiked matrix models to inhomogeneous noise with general variance profiles, establishing computational lower bounds via low-degree polynomial analysis and sharp information-theoretic limits that hold beyond block-structured settings. By introducing and exploiting the reciprocal-variance matrix $oldsymbol{ riangle}^{-1}$ and an AMP-inspired spectral transform $oldsymbol{ moldsymbol{H}}(Y)$, the authors identify a general threshold $eta_*$ that governs detectability and low-rank recovery, and show that for $eta<eta_*$ no efficient procedure (in the low-degree sense) can succeed, while for $eta>eta_*$ certain spectral methods achieve separation. They develop tight bounds for block-structured and general variance profiles, including growth of the low-degree $oldsymbol{ extchi^2}$-divergence and channel-universality arguments, and provide numerical experiments for smoothly varying variance profiles that support the potential optimality of the LinAMP-based spectral algorithm. The results suggest a broad computational threshold governing inhomogeneous noise models and offer tools (graph-sum bounds, partition-based analyses) that may be of independent interest in random matrix theory and beyond. Overall, the paper advances our understanding of statistical-to-computational gaps in high-dimensional low-rank estimation under realistic, nonuniform noise.

Abstract

Recent work has generalized several results concerning the well-understood spiked Wigner matrix model of a low-rank signal matrix corrupted by additive i.i.d. Gaussian noise to the inhomogeneous case, where the noise has a variance profile. In particular, for the special case where the variance profile has a block structure, a series of results identified an effective spectral algorithm for detecting and estimating the signal, identified the threshold signal strength required for that algorithm to succeed, and proved information-theoretic lower bounds that, for some special signal distributions, match the above threshold. We complement these results by studying the computational optimality of this spectral algorithm. Namely, we show that, for a much broader range of signal distributions, whenever the spectral algorithm cannot detect a low-rank signal, then neither can any low-degree polynomial algorithm. This gives the first evidence for a computational hardness conjecture of Guionnet, Ko, Krzakala, and Zdeborová (2023). With similar techniques, we also prove sharp information-theoretic lower bounds for a class of signal distributions not treated by prior work. Unlike all of the above results on inhomogeneous models, our results do not assume that the variance profile has a block structure, and suggest that the same spectral algorithm might remain optimal for quite general profiles. We include a numerical study of this claim for an example of a smoothly-varying rather than piecewise-constant profile. Our proofs involve analyzing the graph sums of a matrix, which also appear in free and traffic probability, but we require new bounds on these quantities that are tighter than existing ones for non-negative matrices, which may be of independent interest.

Computational and statistical lower bounds for low-rank estimation under general inhomogeneous noise

TL;DR

This work extends the theory of spiked matrix models to inhomogeneous noise with general variance profiles, establishing computational lower bounds via low-degree polynomial analysis and sharp information-theoretic limits that hold beyond block-structured settings. By introducing and exploiting the reciprocal-variance matrix and an AMP-inspired spectral transform , the authors identify a general threshold that governs detectability and low-rank recovery, and show that for no efficient procedure (in the low-degree sense) can succeed, while for certain spectral methods achieve separation. They develop tight bounds for block-structured and general variance profiles, including growth of the low-degree -divergence and channel-universality arguments, and provide numerical experiments for smoothly varying variance profiles that support the potential optimality of the LinAMP-based spectral algorithm. The results suggest a broad computational threshold governing inhomogeneous noise models and offer tools (graph-sum bounds, partition-based analyses) that may be of independent interest in random matrix theory and beyond. Overall, the paper advances our understanding of statistical-to-computational gaps in high-dimensional low-rank estimation under realistic, nonuniform noise.

Abstract

Recent work has generalized several results concerning the well-understood spiked Wigner matrix model of a low-rank signal matrix corrupted by additive i.i.d. Gaussian noise to the inhomogeneous case, where the noise has a variance profile. In particular, for the special case where the variance profile has a block structure, a series of results identified an effective spectral algorithm for detecting and estimating the signal, identified the threshold signal strength required for that algorithm to succeed, and proved information-theoretic lower bounds that, for some special signal distributions, match the above threshold. We complement these results by studying the computational optimality of this spectral algorithm. Namely, we show that, for a much broader range of signal distributions, whenever the spectral algorithm cannot detect a low-rank signal, then neither can any low-degree polynomial algorithm. This gives the first evidence for a computational hardness conjecture of Guionnet, Ko, Krzakala, and Zdeborová (2023). With similar techniques, we also prove sharp information-theoretic lower bounds for a class of signal distributions not treated by prior work. Unlike all of the above results on inhomogeneous models, our results do not assume that the variance profile has a block structure, and suggest that the same spectral algorithm might remain optimal for quite general profiles. We include a numerical study of this claim for an example of a smoothly-varying rather than piecewise-constant profile. Our proofs involve analyzing the graph sums of a matrix, which also appear in free and traffic probability, but we require new bounds on these quantities that are tighter than existing ones for non-negative matrices, which may be of independent interest.

Paper Structure

This paper contains 42 sections, 41 theorems, 166 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Spiked Matrix Models
  3. Inhomogeneous Spiked Matrix Models
  4. Main Results
  5. Additional Results
  6. Growth of Low-Degree Chi-Squared Divergence
  7. Statistical-to-Computational Gaps
  8. Channel Universality
  9. Related Work
  10. Spiked matrix models
  11. Inhomogeneous spiked matrix models
  12. Inhomogeneity in random matrix theory
  13. Low-degree polynomial algorithms
  14. Notation
  15. Basic notions
  16. ...and 27 more sections

Key Result

Theorem 1.3

In the above setting, suppose that $\|x\| / \sqrt{N} \xrightarrow{\mathbb{P}} 1$. Then, the following hold.

Figures (3)

  • Figure 1: The graph $G(\pi)$ represented by the partition $\pi =\{\{1,1'\}, \{2,3\}, \{3',4\}, \{4',2'\}\}$ of the set $\mathcal{I} = \{1, 1', 2, 2', 3, 3', 4, 4'\}$.
  • Figure 2: An illustration of the variance profile matrix $\Delta$, plotted as a heatmap of an $N \times N$ matrix with $N = 12000.0$, defined by \ref{['eq:smooth-profile']} with the choice of the function $f$ given in \ref{['eq:profile-f']}. For such large $N$, this heatmap is visually indistinguishable from a plot of the continuous function $f$ itself on the domain $[0, 1]^2$.
  • Figure 3: We compare the direct, whitening, and linearized AMP (LinAMP) spectral algorithms discussed in Section \ref{['sec:numerical']} for several signal strengths $\beta_1 < \beta_2 < \beta_3 < \beta_4$ and $N = 12000.0$, plotting the eigenvalue distribution of the matrix each constructs with the maximum eigenvalue indicated with an arrow. As predicted, the LinAMP algorithm requires the smallest threshold signal strength to observe an outlier eigenvalue, the whitening algorithm the next-smallest strength, and the direct algorithm the largest threshold strength.

Theorems & Definitions (74)

  • Definition 1.1: Strong detection
  • Definition 1.2: Strong separation
  • Theorem 1.3: FP-2007-LargestEigenvalueWignerCDMF-2009-DeformedWigner
  • Theorem 1.4: PWBM-2018-PCAI
  • Theorem 1.5: kunisky2019notes
  • Definition 1.6: Inhomogeneous spiked Wigner matrix model
  • Definition 1.7: Block-structured matrices
  • Theorem 1.8: MKK-2024-OptimalPCABlockSpikedMatrixBCSVH-2024-MatrixConcentrationFreeProbability2
  • Proposition 1.9
  • Theorem 1.10: Block-structured variance profile
  • ...and 64 more