High-dimensional Bayesian inference via the Unadjusted Langevin Algorithm

TL;DR

This work analyzes high-dimensional Bayesian sampling via the Unadjusted Langevin Algorithm (ULA) for targets with density $\pi(x) \propto e^{-U(x)}$ on $\mathbb{R}^d$, under smoothness and strong convexity of $U$. It derives explicit non-asymptotic bounds in Wasserstein-2 and total variation for both constant and decreasing step sizes, with detailed dimension dependence and convergence properties of weighted empirical estimators. The results cover continuous-time Langevin diffusion approximations, discrete-time Euler schemes, and their coupling-based analyses, including TV contractions and MSE/concentration bounds; they also provide practical Bayesian logistic-regression experiments demonstrating competitive performance against established samplers. A separate contraction framework for functional autoregressive models broadens the applicability of the coupling techniques. Collectively, the paper offers precise guidance on step-size schedules, preconditioning, and diagnostic bounds for high-dimensional Bayesian sampling with Langevin-type dynamics.

Abstract

We consider in this paper the problem of sampling a high-dimensional probability distribution $π$ having a density with respect to the Lebesgue measure on $\mathbb{R}^d$, known up to a normalization constant $x \mapsto π(x)= \mathrm{e}^{-U(x)}/\int_{\mathbb{R}^d} \mathrm{e}^{-U(y)} \mathrm{d} y$. Such problem naturally occurs for example in Bayesian inference and machine learning. Under the assumption that $U$ is continuously differentiable, $\nabla U$ is globally Lipschitz and $U$ is strongly convex, we obtain non-asymptotic bounds for the convergence to stationarity in Wasserstein distance of order $2$ and total variation distance of the sampling method based on the Euler discretization of the Langevin stochastic differential equation, for both constant and decreasing step sizes. The dependence on the dimension of the state space of these bounds is explicit. The convergence of an appropriately weighted empirical measure is also investigated and bounds for the mean square error and exponential deviation inequality are reported for functions which are measurable and bounded. An illustration to Bayesian inference for binary regression is presented to support our claims.

Figures (3)

• Figure 1: Plots of $(u_{0,n}^{(4)})_{n \geq 1}\Gamma_{n}$ for four sequences of step sizes $(\gamma_k)_{k \geq 1}$, $\gamma_k = (1+k)^{-\alpha}$ for $\alpha =0,1/4,1/2,3/4$
• Figure 2: Empirical distribution comparison between the Polya-Gamma Gibbs Sampler and ULA. Left panel: constant step size $\gamma_k = \gamma_1$ for all $k \geq 1$; right panel: decreasing step size $\gamma_k = \gamma_1 k^{-1/2}$ for all $k \geq 1$
• Figure 3: Marginal accuracy across all the dimensions. Upper left: German credit data set. Upper right: Australian credit data set. Lower left: Heart disease data set. Lower right: Pima Indian diabetes data set. At the bottom: Musk data set

Theorems & Definitions (68)

• Proposition 1
• proof
• Proposition 2
• proof
• Proposition 3
• proof
• Corollary 1
• proof
• Theorem 1
• proof