Kinetic Langevin Splitting Schemes for Constrained Sampling

Abstract

Constrained sampling is an important and challenging task in computational statistics, concerned with generating samples from a distribution under certain constraints. There are numerous types of algorithm aimed at this task, ranging from general Markov chain Monte Carlo, to unadjusted Langevin methods. In this article we propose a series of new sampling algorithms based on the latter of these, specifically the kinetic Langevin dynamics. Our series of algorithms are motivated on advanced numerical methods which are splitting order schemes, which include the BU and BAO families of splitting schemes.Their advantage lies in the fact that they have favorable strong order (bias) rates and computationally efficiency. In particular we provide a number of theoretical insights which include a Wasserstein contraction and convergence results. We are able to demonstrate favorable results, such as improved complexity bounds over existing non-splitting methodologies. Our results are verified through numerical experiments on a range of models with constraints, which include a toy example and Bayesian linear regression.

Figures (8)

• Figure 1: Comparison between different constraint algorithms, when aiming to sample from the circular convex set $\mathcal{K}$ defined by $\mathcal{K} = \mathcal{B}_2(0,R)$. We consider $n=1000$ iterations and $2500$ time steps.
• Figure 2: Comparison between different constraint algorithms with stochastic gradients, when aiming to sample from the circular convex set $\mathcal{K}$ defined by $\mathcal{K} = \mathcal{B}_2(0,R)$. We consider $n=1000$ iterations and $2500$ time steps.
• Figure 3: Comparison between different constraint algorithms, when aiming to sample from the triangular convex set $\mathcal{K}$ defined in Eqn. \ref{['eqn:convex_set_t']}. We consider $n=1000$ iterations and $2500$ time steps.
• Figure 4: Comparison between different constraint algorithms with stochastic gradients, when aiming to sample from the triangular convex set $\mathcal{K}$ defined in Eqn. \ref{['eqn:convex_set_t']} . We consider $n=1000$ iterations and $2500$ time steps.
• Figure 5: Comparison between different constraint algorithms, when aiming to sample from the square convex set $\mathcal{K}$ defined in Eqn. \ref{['eqn:convex_set_s']} . We consider $n=1000$ iterations and $2500$ time steps.

Theorems & Definitions (33)

• Example 2.1: Bregman projection
• Example 2.2: Scaling-based projection
• Example 2.3: Ellipsoid
• Example 2.4: $\ell_q$-ball with $q>2$
• Theorem 2.1
• Remark 2.2
• Definition 2.1
• Definition 2.2: $q$-Wasserstein distance
• Proposition 2.1
• Remark 2.3