Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Impossibility of latent inner product recovery via rate distortion

Cheng Mao, Shenduo Zhang

TL;DR

This work establishes an information-theoretic impossibility of recovering latent inner products in latent-space random geometric graphs when the latent dimension satisfies $d \gtrsim n h(p)$, aligning the recovery threshold with entropy-based predictions. The authors develop a lower bound on the rate-distortion function for the Wishart distribution $X=ZZ^\top$ with isotropic Gaussian or spherical rows, and use rate-distortion theory together with data-processing and entropy arguments to show that accurate inner-product recovery is impossible beyond the threshold. The approach also yields corollaries, including an impossibility result for one-bit matrix completion under a Wishart prior, and provides a rigorous framework linking latent geometry, rate-distortion, and information-theoretic limits. The results clarify when geometry in latent spaces cannot be inferred from sparse binary observations and establish a sharp recovery boundary that complements existing positive results.)

Abstract

In this largely expository note, we present an impossibility result for inner product recovery in a random geometric graph or latent space model using the rate-distortion theory. More precisely, suppose that we observe a graph $A$ on $n$ vertices with average edge density $p$ generated from Gaussian or spherical latent locations $z_1, \dots, z_n \in \mathbb{R}^d$ associated with the $n$ vertices. It is of interest to estimate the inner products $\langle z_i, z_j \rangle$ which represent the geometry of the latent points. We prove that it is impossible to recover the inner products if $d \gtrsim n h(p)$ where $h(p)$ is the binary entropy function. This matches the condition required for positive results on inner product recovery in the literature. The proof follows the well-established rate-distortion theory with the main technical ingredient being a lower bound on the rate-distortion function of the Wishart distribution which is interesting in its own right.

Impossibility of latent inner product recovery via rate distortion

TL;DR

This work establishes an information-theoretic impossibility of recovering latent inner products in latent-space random geometric graphs when the latent dimension satisfies , aligning the recovery threshold with entropy-based predictions. The authors develop a lower bound on the rate-distortion function for the Wishart distribution with isotropic Gaussian or spherical rows, and use rate-distortion theory together with data-processing and entropy arguments to show that accurate inner-product recovery is impossible beyond the threshold. The approach also yields corollaries, including an impossibility result for one-bit matrix completion under a Wishart prior, and provides a rigorous framework linking latent geometry, rate-distortion, and information-theoretic limits. The results clarify when geometry in latent spaces cannot be inferred from sparse binary observations and establish a sharp recovery boundary that complements existing positive results.)

Abstract

In this largely expository note, we present an impossibility result for inner product recovery in a random geometric graph or latent space model using the rate-distortion theory. More precisely, suppose that we observe a graph on vertices with average edge density generated from Gaussian or spherical latent locations associated with the vertices. It is of interest to estimate the inner products which represent the geometry of the latent points. We prove that it is impossible to recover the inner products if where is the binary entropy function. This matches the condition required for positive results on inner product recovery in the literature. The proof follows the well-established rate-distortion theory with the main technical ingredient being a lower bound on the rate-distortion function of the Wishart distribution which is interesting in its own right.
Paper Structure (8 sections, 8 theorems, 58 equations)

This paper contains 8 sections, 8 theorems, 58 equations.

Table of Contents

  1. Introduction
  2. Main results
  3. Open problems.
  4. Proof of Theorem \ref{['thm:wishart-rate-distortion']}
  5. Case $d \le c^* n$
  6. Case $d \geq n$
  7. Case $c^* n < d < n$
  8. Spherical case

Key Result

Theorem 2.2

For positive integers $n$ and $d$, let $Z := [z_1 \dots z_n]^\top \in \mathbb{R}^{n \times d}$ where the i.i.d. rows $z_1, \dots, z_n$ follow either the Gaussian distribution $\mathcal{N}(0, \frac{1}{d} I_d)$ or the uniform distribution on the unit sphere $\mathcal{S}^{d-1} \subset \mathbb{R}^d$. Le Let $n \land d := \min\{n,d\}$. There is an absolute constant $c>0$ such that for any $D \in (0, c)

Theorems & Definitions (14)

  • Definition 2.1: Rate-distortion function
  • Theorem 2.2: Rate-distortion function of a Wishart matrix
  • Corollary 2.3: Random geometric graph or latent space model
  • proof
  • Corollary 2.4: One-bit matrix completion
  • proof
  • Lemma 3.1
  • proof
  • Lemma 3.2
  • proof
  • ...and 4 more