Hadamard Langevin dynamics for sampling the l1-prior

TL;DR

The paper introduces Hadamard-Langevin dynamics (HLD) to sample from sparsity-promoting posteriors with non-smooth log-densities, by lifting the problem to a Hadamard overparameterization $(u,v)$ so that $x=u\odot v$ recovers the target posterior $\rho$. It proves a rigorous well-posedness theory for both the continuous diffusion and its discretization, including geometric ergodicity and convergence of the discretized scheme as the time step vanishes. Two analytical routes are developed to handle the non-smooth log singularities: a Cartesian reparameterization that regularizes the drift, and a Girsanov transformation linking the dynamics to CIR and OU processes; the work also connects HLD to Riemannian Langevin diffusion. The results establish the first theoretical foundation for sampling from nonconvex, nonsmooth posteriors via overparameterized Langevin dynamics and provide convergence guarantees for the discretization, with discussion of extensions to broader sparse priors. Practical implications include unbiased sampling from Laplace-type priors without smoothing biases, with potential impact on Bayesian inverse problems and high-dimensional sparsity-promoting inference.

Abstract

Priors with non-smooth log-densities, such as the l1-prior, are widely used in Bayesian inverse problems for their sparsity-inducing properties. Existing Langevin-based sampling methods typically rely on proximal mappings or smooth approximations, which alter the target distribution. We propose an alternative approach based on a Hadamard product parameterization of the l1-norm, leading to a smooth but nonconvex and non-globally Lipschitz potential whose marginal law exactly recovers the desired posterior. The resulting Hadamard Langevin dynamics (HLD) defines a diffusion process that is analytically distinct from proximal or mirror-type Langevin schemes. Our main contribution is a rigorous well-posedness theory for both the continuous and discrete HLD. We establish existence and uniqueness of strong solutions, geometric ergodicity of the continuous dynamics, and convergence of the discretized scheme as the step size tends to zero. These results provide the first theoretical foundation for sampling from nonconvex, nonsmooth posteriors through overparameterized Langevin dynamics.

Figures (5)

• Figure 1: Estimation of the convergence rate for a 1D problem. (a) Error of the 'stationary distribution' for different stepsizes. (b) Error decay with increasing iterations for fixed stepsize.
• Figure 2: Plots of sample means against iteration. The right figure shows the ESS in log-scale at of the 20 dimensions. The minimum ESS (in log scale) for Prox-l1 is 4.0, for Hadamard is 6.4 and for Gibbs is 9.2.
• Figure 3: 1d Haar wavelet deconvolution. (a) the mode and mean signals. (b) the difference between the 95 quantile and the 5 quantile. (c) the ESS at each pixel. The minimum ESS across all pixels for Hadamard is 286,433.7 while the minimum ESS for Prox-l1 is 4.1.
• Figure 4: Observation and mode image.
• Figure 5: Mean images and log differences between 95 and 5 quantiles for different choices of $\beta$.

Theorems & Definitions (46)

• Theorem 1.1
• Theorem 2.1
• proof
• Proposition 2.2
• proof
• Corollary 2.3
• proof
• Proposition 2.4
• proof
• Lemma 2.5: Novikov condition