Dynamical mean-field analysis of adaptive Langevin diffusions: Propagation-of-chaos and convergence of the linear response

TL;DR

This work establishes a rigorous dynamical mean-field theory (DMFT) framework for adaptive Langevin diffusions with random disorder. It proves the existence and uniqueness of a DMFT fixed point and shows that, in the large-n,d limit, the empirical laws of coordinates and the adapting drift converge to deterministic limit processes described by self-consistent correlation and response kernels. A dynamical cavity argument demonstrates that these kernels arise as mean-field limits of single-coordinate observables, enabling a precise propagation-of-chaos result with coordinate-wise decoupling. The three-step program—discretization, continuous-time limit, and Langevin discretization—yields a complete convergence theory for the adaptive Langevin system, including the convergence of linear response functions. The findings pave the way for analyzing empirical Bayes Langevin dynamics and related high-dimensional posterior-sampling procedures in companion work, by providing a principled DMFT description of the learning dynamics.

Abstract

Motivated by an application to empirical Bayes learning in high-dimensional regression, we study a class of Langevin diffusions in a system with random disorder, where the drift coefficient is driven by a parameter that continuously adapts to the empirical distribution of the realized process up to the current time. The resulting dynamics take the form of a stochastic interacting particle system having both a McKean-Vlasov type interaction and a pairwise interaction defined by the random disorder. We prove a propagation-of-chaos result, showing that in the large system limit over dimension-independent time horizons, the empirical distribution of sample paths of the Langevin process converges to a deterministic limit law that is described by dynamical mean-field theory. This law is characterized by a system of dynamical fixed-point equations for the limit of the drift parameter and for the correlation and response kernels of the limiting dynamics. Using a dynamical cavity argument, we verify that these correlation and response kernels arise as the asymptotic limits of the averaged correlation and linear response functions of single coordinates of the system. These results enable an asymptotic analysis of an empirical Bayes Langevin dynamics procedure for learning an unknown prior parameter in a linear regression model, which we develop in a companion paper.

Theorems & Definitions (67)

• Theorem 2.4
• Theorem 2.5
• Corollary 2.6
• proof
• Corollary 2.7
• proof
• Theorem 2.8
• Remark 2.9
• Lemma 3.1
• proof