Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Low-rank Matrix Estimation with Inhomogeneous Noise

Alice Guionnet, Justin Ko, Florent Krzakala, Lenka Zdeborová

TL;DR

This work addresses low-rank matrix estimation under inhomogeneous noise by establishing a Gaussian universality principle that maps inhomogeneous models to Gaussian spike models with a variance profile. It provides a rigorous large-$N$ free energy formula via replica methods, analyzes spectral BBP-type transitions, and derives information-theoretic detectability thresholds for inhomogeneous settings, including the degree-corrected stochastic block model. The results unify the treatment of diverse noise profiles, link free-energy to MMSE, and reveal when spectral methods are optimal or suboptimal. The framework has direct implications for community detection in networks with heterogeneous noise, offering precise threshold criteria and a tractable variational problem to compute recovery limits. Overall, the paper extends the spiked-model methodology to inhomogeneous outputs, delivering both theoretical universality results and practical insights for structured estimation tasks.

Abstract

We study low-rank matrix estimation for a generic inhomogeneous output channel through which the matrix is observed. This generalizes the commonly considered spiked matrix model with homogeneous noise to include for instance the dense degree-corrected stochastic block model. We adapt techniques used to study multispecies spin glasses to derive and rigorously prove an expression for the free energy of the problem in the large size limit, providing a framework to study the signal detection thresholds. We discuss an application of this framework to the degree corrected stochastic block models.

Low-rank Matrix Estimation with Inhomogeneous Noise

TL;DR

This work addresses low-rank matrix estimation under inhomogeneous noise by establishing a Gaussian universality principle that maps inhomogeneous models to Gaussian spike models with a variance profile. It provides a rigorous large- free energy formula via replica methods, analyzes spectral BBP-type transitions, and derives information-theoretic detectability thresholds for inhomogeneous settings, including the degree-corrected stochastic block model. The results unify the treatment of diverse noise profiles, link free-energy to MMSE, and reveal when spectral methods are optimal or suboptimal. The framework has direct implications for community detection in networks with heterogeneous noise, offering precise threshold criteria and a tractable variational problem to compute recovery limits. Overall, the paper extends the spiked-model methodology to inhomogeneous outputs, delivering both theoretical universality results and practical insights for structured estimation tasks.

Abstract

We study low-rank matrix estimation for a generic inhomogeneous output channel through which the matrix is observed. This generalizes the commonly considered spiked matrix model with homogeneous noise to include for instance the dense degree-corrected stochastic block model. We adapt techniques used to study multispecies spin glasses to derive and rigorously prove an expression for the free energy of the problem in the large size limit, providing a framework to study the signal detection thresholds. We discuss an application of this framework to the degree corrected stochastic block models.
Paper Structure (25 sections, 36 theorems, 165 equations, 6 figures)

This paper contains 25 sections, 36 theorems, 165 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Highlights of our main contributions
  3. Relation with previous work
  4. Main results
  5. Setting
  6. Gaussian Estimation Problems with Covariance Profiles
  7. Effective data matrix
  8. Equivalent spike model
  9. The Limit of the Free Energy and Consequences
  10. The BBP Transition of a Transformed Homogeneous Noise Model
  11. Application to the Degree Corrected Stochastic Block Model
  12. The Limit of the Free Energy
  13. Rademacher Prior
  14. Outline of the Paper.
  15. Universality in Inhomogenous Vector Spin Models
  16. ...and 10 more sections

Key Result

Lemma 2.6

Suppose that Hypothesis hypg and Hypothesis hypcompact are satisfied. Let $B$ be a $\sigma$ algebra such that the $D_{ij}$ are independent conditionally to $B$. Then with

Figures (6)

  • Figure 1: The free energy as a function of the noise. A case with a continuous phase transition (dashed line) separating the undetectable regime from one where detection of the signal is possible.
  • Figure 2: The free energy as a function of the noise. A case with a discontinuous phase transition (orange dashed line) separating the undetectable regime from one where detection of the signal is possible. We plots the free energy of two local maximizers (in blue and red), it is the larger one that provides the final result. The purple dashed line is a position of the threshold from part 2 of Lemma \ref{['prop:recovery']} that is not tight in this case.
  • Figure 3: The free energy as a function of the noise parameter, for groups of different sizes $p$, and diagonal inhomogeneity $\Delta$.
  • Figure 4: The free energy as a function of the noise parameter, for groups of different sizes $p$, and nondiagonal inhomogeneity $\Delta$.
  • Figure 5: Free energy as the function of noise for the DCSBM with a small size of one of the groups. We see a discontinuous phase transition (purple line).
  • ...and 1 more figures

Theorems & Definitions (52)

  • Remark 2.5
  • Lemma 2.6: Free Energy Universality 1
  • Theorem 2.7: Free Energy Universality 2
  • Remark 2.8
  • Theorem 2.9: Universality of the Spectrum
  • Theorem 2.10: Bayes Optimal Free Energy
  • Remark 2.12
  • Theorem 2.13: Bayes Optimal Free Energy with General Kernel
  • Corollary 2.14: Limiting MMSE
  • Lemma 2.15: Recovery Transitions
  • ...and 42 more