Constrained and Composite Sampling via Proximal Sampler

TL;DR

This paper develops a unified framework for constrained and composite log-concave sampling via epigraph lifting, transforming feasibility constraints into nearly uniform sampling on lifted convex bodies. It leverages a proximal sampler with a restricted Gaussian oracle (RGO) implemented through cutting-plane methods and rejection sampling, requiring only separation and subgradient/proximal oracles. The approach yields unbiased samples and provides mixing-time guarantees in Rényi and χ^2 divergences, with explicit iteration and oracle complexities under standard Lipschitz and warm-start assumptions. By keeping f and h separate in the composite setting, the authors show how different oracle accesses can construct separation oracles for the lifted problem, enabling practical and geometry-free constrained and composite sampling for Bayesian-like models. Overall, the work advances sampling-optimization interplay and offers practical, theoretically-grounded tools for high-dimensional log-concave sampling with constraints and composite objectives.

Abstract

We study two log-concave sampling problems: constrained sampling and composite sampling. First, we consider sampling from a target distribution with density proportional to $\exp(-f(x))$ supported on a convex set $K \subset \mathbb{R}^d$, where $f$ is convex. The main challenge is enforcing feasibility without degrading mixing. Using an epigraph transformation, we reduce this task to sampling from a nearly uniform distribution over a lifted convex set in $\mathbb{R}^{d+1}$. We then solve the lifted problem using a proximal sampler. Assuming only a separation oracle for $K$ and a subgradient oracle for $f$, we develop an implementation of the proximal sampler based on the cutting-plane method and rejection sampling. Unlike existing constrained samplers that rely on projection, reflection, barrier functions, or mirror maps, our approach enforces feasibility using only minimal oracle access, resulting in a practical and unbiased sampler without knowing the geometry of the constraint set. Second, we study composite sampling, where the target is proportional to $\exp(-f(x)-h(x))$ with closed and convex $f$ and $h$. This composite structure is standard in Bayesian inference with $f$ modeling data fidelity and $h$ encoding prior information. We reduce composite sampling via an epigraph lifting of $h$ to constrained sampling in $\mathbb{R}^{d+1}$, which allows direct application of the constrained sampling algorithm developed in the first part. This reduction results in a double epigraph lifting formulation in $\mathbb{R}^{d+2}$, on which we apply a proximal sampler. By keeping $f$ and $h$ separate, we further demonstrate how different combinations of oracle access (such as subgradient and proximal) can be leveraged to construct separation oracles for the lifted problem. For both sampling problems, we establish mixing time bounds measured in Rényi and $χ^2$ divergences.

Theorems & Definitions (61)

• Theorem 3.1
• Lemma 3.2
• Theorem 3.3
• Lemma 3.4
• Lemma 3.5
• Theorem 4.1
• Theorem 4.2
• Proposition B.1
• Lemma B.2
• proof