Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High-dimensional Asymptotics of Langevin Dynamics in Spiked Matrix Models

Tengyuan Liang, Subhabrata Sen, Pragya Sur

TL;DR

The paper analyzes Langevin dynamics for recovering a planted signal in a high-dimensional spiked matrix model, providing a path-wise overlap description via CHSCK equations. It proves that, in the large-dimension limit, the overlap and correlation functions converge to deterministic CHSCK dynamics and derives an explicit long-time overlap formula in a semi-circle noise setting, revealing a sharp phase transition controlled by the SNR $ Lambda$ and diffusion strength $eta^{-1}$. The transition threshold involves the modified signal strength $ ilde{ Lambda}= Lambda+ rac{ Sigma_*^2}{4 Lambda}$ and a root $s_eta$ of the Stieltjes transform equation, with nonzero limiting correlation occurring when $2 ilde{ Lambda}>s_eta$ and the initialization correlation $ ho$ is positive. These results rigorously connect dynamical mean-field theory to planted-inference models and quantify the effect of early stopping and injected noise on the recovery performance.

Abstract

We study Langevin dynamics for recovering the planted signal in the spiked matrix model. We provide a "path-wise" characterization of the overlap between the output of the Langevin algorithm and the planted signal. This overlap is characterized in terms of a self-consistent system of integro-differential equations, usually referred to as the Crisanti-Horner-Sommers-Cugliandolo-Kurchan (CHSCK) equations in the spin glass literature. As a second contribution, we derive an explicit formula for the limiting overlap in terms of the signal-to-noise ratio and the injected noise in the diffusion. This uncovers a sharp phase transition -- in one regime, the limiting overlap is strictly positive, while in the other, the injected noise overcomes the signal, and the limiting overlap is zero.

High-dimensional Asymptotics of Langevin Dynamics in Spiked Matrix Models

TL;DR

The paper analyzes Langevin dynamics for recovering a planted signal in a high-dimensional spiked matrix model, providing a path-wise overlap description via CHSCK equations. It proves that, in the large-dimension limit, the overlap and correlation functions converge to deterministic CHSCK dynamics and derives an explicit long-time overlap formula in a semi-circle noise setting, revealing a sharp phase transition controlled by the SNR and diffusion strength . The transition threshold involves the modified signal strength and a root of the Stieltjes transform equation, with nonzero limiting correlation occurring when and the initialization correlation is positive. These results rigorously connect dynamical mean-field theory to planted-inference models and quantify the effect of early stopping and injected noise on the recovery performance.

Abstract

We study Langevin dynamics for recovering the planted signal in the spiked matrix model. We provide a "path-wise" characterization of the overlap between the output of the Langevin algorithm and the planted signal. This overlap is characterized in terms of a self-consistent system of integro-differential equations, usually referred to as the Crisanti-Horner-Sommers-Cugliandolo-Kurchan (CHSCK) equations in the spin glass literature. As a second contribution, we derive an explicit formula for the limiting overlap in terms of the signal-to-noise ratio and the injected noise in the diffusion. This uncovers a sharp phase transition -- in one regime, the limiting overlap is strictly positive, while in the other, the injected noise overcomes the signal, and the limiting overlap is zero.
Paper Structure (5 sections, 11 theorems, 126 equations, 1 figure)

This paper contains 5 sections, 11 theorems, 126 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Related literature
  3. Proof of Theorem \ref{['thm:limit_value']}
  4. Proof of Theorem \ref{['thm:limits']}
  5. Proof of Lemma \ref{['lemma:deterministic']}

Key Result

Theorem 1.1

Assume one of the Initial Conditions specified above. Fix $T\geq 0$. As $N \to \infty$, $R^N$ and $K^N$ converge almost surely to deterministic limits $R$ and $K$ respectively. Furthermore, these limits are the unique solutions to the following system of integro-differential equations: Here we use the abbreviated notation $K(t):= K(t,t)$. It remains to specify the distribution $\pi^{\infty}$; thi

Figures (1)

  • Figure 1: The contour integral for evaluating the Fourier-Mellin formula.

Theorems & Definitions (25)

  • Remark 1
  • Remark 2
  • Theorem 1.1
  • Theorem 1.2
  • Lemma 2.1
  • proof : Proof of Lemma \ref{['lemma:g_inversion']}
  • Lemma 2.2
  • proof : Proof of Lemma \ref{['lem:h_laplace']}
  • Lemma 2.3
  • proof
  • ...and 15 more