Regularized Dikin Walks for Sampling Truncated Logconcave Measures, Mixed Isoperimetry and Beyond Worst-Case Analysis

TL;DR

This work tackles the challenge of sampling from truncated logconcave distributions on polytopes, a problem central to Bayesian inference with indicator variables and related applications. It introduces regularized Dikin walks that combine a base Hessian-like metric with a fixed regularizer, enabling efficient Metropolis-corrected proposals from curved local ellipsoids; two concrete metrics—soft-threshold and regularized Lewis—exploit polytope structure. The authors prove mixing-time guarantees in both strongly and weakly logconcave settings, deriving tilde-notation bounds such as $\widetilde{O}((m+\kappa)n)$ for strongly logconcave targets and $\widetilde{O}(n^{2.5}+\kappa n)$ for Lewis-weight-based variants, plus extensions to finite-covariance (weakly) targets and beyond-worst-case analyses that depend on how many constraints intersect the high-probability region. A key technical contribution is a new isoperimetric inequality that blends Euclidean and Hilbert (cross-ratio) distances, enabling conductance-based mixing-time proofs; stochastic localization is used to extend results to weakly logconcave measures. Practical aspects are discussed, including per-iteration costs, warm-start constructions, and the potential for applying these methods to SUN/posterior sampling problems in Bayesian probit and related models. Overall, the paper advances fast, structure-exploiting sampling from constrained logconcave distributions with meaningful implications for high-dimensional Bayesian computation and volume-related tasks.

Abstract

We study the problem of drawing samples from a logconcave distribution truncated on a polytope, motivated by computational challenges in Bayesian statistical models with indicator variables, such as probit regression. Building on interior point methods and the Dikin walk for sampling from uniform distributions, we analyze the mixing time of regularized Dikin walks. Our contributions are threefold. First, for a logconcave and log-smooth distribution with condition number $κ$, truncated on a polytope in $\mathbb{R}^n$ defined with $m$ linear constraints, we prove that the soft-threshold Dikin walk mixes in $\widetilde{O}((m+κ)n)$ iterations from a warm initialization. It improves upon prior work which required the polytope to be bounded and involved a bound dependent on the radius of the bounded region. Moreover, we introduce the regularized Dikin walk using Lewis weights for approximating the John ellipsoid. We show that it mixes in $\widetilde{O}((n^{2.5}+κn)$. Second, we extend the mixing time guarantees mentioned above to weakly log-concave distributions truncated on polytopes, provided that they have a finite covariance matrix. Third, going beyond worst-case mixing time analysis, we demonstrate that soft-threshold Dikin walk can mix significantly faster when only a limited number of constraints intersect the high-probability mass of the distribution, improving the $\widetilde{O}((m+κ)n)$ upper bound to $\widetilde{O}(m + κn)$. Additionally, per-iteration complexity of regularized Dikin walk and ways to generate a warm initialization are discussed to facilitate practical implementation.

Figures (2)

• Figure 1: An example of Theorem \ref{['th_prob_ball_intersection']} in $\mathbb{R}^2$ ($n=2$), where $K$ is the polytope and the ball $\mathcal{B}_\Upsilon^\delta$ refers to the high-probability ball of the truncated distribution $\Pi$. The dashed ellipsoids are contours for the potential $f(x)$ of the distribution. The dashed segments are the constraints that are violated, the solid segments are untouched constraints, so $\mathcal{M}_\Upsilon^\delta=2$, $m=6$.
• Figure 2: An example of warm-start $\mathbb{B}(x_0,r_0)$ for $K\subseteq \mathbb{R}^2$: Here the mode within the polytope $x^\dag:= \arg\min_K f(x)$ coincides the upper-left vertex of $K$. We need to ensure $\mathbb{B}(x_0,r_0)\subseteq \mathbb{B}(x^\dag,r_1)$ and $\mathbb{B}(x_0,r_0)\subseteq K$, where the first condition reduces to $\left\| x_0-x^\dag\right\|_{2}+r_0\leq r_1$, and the later is ensured by $\mathbb{B}(x_0,r_0)$ being inside the convex hull of $\{x^\dag\}\cup\mathbb{B}(x_1,\widetilde{r})$, and $r_0$ can be easily computed by similarity of cones.

Theorems & Definitions (56)

• Definition 1: soft-threshold NEURIPS2023_mangoubi
• Definition 2: regularized Lewis metric
• Definition 3
• Definition 4: laddha2020strongpmlr-v247-kook24b
• Theorem 1
• Corollary 1: soft-threshold metric, strongly logconcave target
• Corollary 2: soft-threshold metric, Gaussian target
• Corollary 3: regularized Lewis metric, strongly logconcave target
• Theorem 2
• Corollary 4: soft-threshold metric, weakly logconcave target