Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Faster logconcave sampling from a cold start in high dimension

Yunbum Kook, Santosh S. Vempala

TL;DR

The paper presents sub-cubic algorithms for sampling arbitrary log-concave densities from warm starts in high dimensions by relaxing warmness from $R_ ext{∞}$ to $R_c$ ($c= ilde{O}(1)$) and by refining the log-Sobolev inequality (LSI) bounds. A key technical advance is a new $C_{ ext{LSI}}( ho) ilde{ o} D \, orm{ ext{cov}( ho)}^{1/2}$ bound and a result showing $ orm{ ext{cov}( ho)\nabla^2 ext{Gauss}} ilde{ o} orm{ ext{cov}( ho)}$ under Gaussian weighting, enabling faster annealing via a proximal sampler and accelerated Gaussian-cooling schemes. Consequently, uniform and Gaussian sampling from convex bodies in near-isotropic position can be achieved in $ ilde{O}(n^{2} R^{3/2} orm{ ext{cov} ho}^{1/4})$ queries, and near-isotropic cases attain $ ilde{O}(n^{2.75})$, with a $n^{2.5}$ bound for truncated Gaussians. The framework extends to general log-concave distributions via a well-defined function oracle, yielding $ ilde{O}(n^{2.5} + n^{2} R^{3/2} orm{ ext{cov} ho}^{1/4})$ queries, thus matching the efficiency of uniform sampling in broader settings and enabling practical warm-start generation for complex targets.

Abstract

We present a faster algorithm to generate a warm start for sampling an arbitrary logconcave density specified by an evaluation oracle, leading to the first sub-cubic sampling algorithms for inputs in (near-)isotropic position. A long line of prior work incurred a warm-start penalty of at least linear in the dimension, hitting a cubic barrier, even for the special case of uniform sampling from convex bodies. Our improvement relies on two key ingredients of independent interest. (1) We show how to sample given a warm start in weaker notions of distance, in particular $q$-Rényi divergence for $q=\widetilde{\mathcal{O}}(1)$, whereas previous analyses required stringent $\infty$-Rényi divergence (with the exception of Hit-and-Run, whose known mixing time is higher). This marks the first improvement in the required warmness since Lovász and Simonovits (1991). (2) We refine and generalize the log-Sobolev inequality of Lee and Vempala (2018), originally established for isotropic logconcave distributions in terms of the diameter of the support, to logconcave distributions in terms of a geometric average of the support diameter and the largest eigenvalue of the covariance matrix.

Faster logconcave sampling from a cold start in high dimension

TL;DR

The paper presents sub-cubic algorithms for sampling arbitrary log-concave densities from warm starts in high dimensions by relaxing warmness from to () and by refining the log-Sobolev inequality (LSI) bounds. A key technical advance is a new bound and a result showing under Gaussian weighting, enabling faster annealing via a proximal sampler and accelerated Gaussian-cooling schemes. Consequently, uniform and Gaussian sampling from convex bodies in near-isotropic position can be achieved in queries, and near-isotropic cases attain , with a bound for truncated Gaussians. The framework extends to general log-concave distributions via a well-defined function oracle, yielding queries, thus matching the efficiency of uniform sampling in broader settings and enabling practical warm-start generation for complex targets.

Abstract

We present a faster algorithm to generate a warm start for sampling an arbitrary logconcave density specified by an evaluation oracle, leading to the first sub-cubic sampling algorithms for inputs in (near-)isotropic position. A long line of prior work incurred a warm-start penalty of at least linear in the dimension, hitting a cubic barrier, even for the special case of uniform sampling from convex bodies. Our improvement relies on two key ingredients of independent interest. (1) We show how to sample given a warm start in weaker notions of distance, in particular -Rényi divergence for , whereas previous analyses required stringent -Rényi divergence (with the exception of Hit-and-Run, whose known mixing time is higher). This marks the first improvement in the required warmness since Lovász and Simonovits (1991). (2) We refine and generalize the log-Sobolev inequality of Lee and Vempala (2018), originally established for isotropic logconcave distributions in terms of the diameter of the support, to logconcave distributions in terms of a geometric average of the support diameter and the largest eigenvalue of the covariance matrix.
Paper Structure (78 sections, 55 theorems, 259 equations, 1 algorithm)

This paper contains 78 sections, 55 theorems, 259 equations, 1 algorithm.

Table of Contents

  1. Introduction
  2. A brief history of sampling and warmth.
  3. Uniform sampling from a warm start.
  4. Warm-start generation by annealing.
  5. Log-Sobolev inequality for strongly logconcave distributions.
  6. Back to warm-start generation.
  7. Extension to logconcave distributions.
  8. Results
  9. Result 1: Uniform and Gaussian sampling under a relaxed warmness (§\ref{['sec:unif-sampling-warm']}).
  10. Result 2: LSI for strongly logconcave distributions with compact support (§\ref{['app:LSI-interpolation-SL']}).
  11. Result 3: Faster sampling from uniform and Gaussian distributions (§\ref{['sec:faster-generation']}).
  12. Result 4: Extension to logconcave distributions (§\ref{['sec:extension']}).
  13. Technical overview
  14. Sampling under weaker warmness
  15. $\mathsf{Ball\ walk}$ under relaxed warmness.
  16. ...and 63 more sections

Key Result

Theorem 1.3

Let $\pi$ be a uniform distribution over a convex body $\mathcal{K}\subset\mathbb{R}^{n}$ specified by $\mathsf{Mem}_{R}(\mathcal{K})$ with $\lambda=\lVert\operatorname{cov}\pi\rVert$. For any $\eta,\varepsilon\in(0,1)$, initial distribution $\pi_{0}$ with $c=\operatorname{polylog}\frac{n\lambda M_{ membership queries in expectation.

Theorems & Definitions (85)

  • Definition 1.1
  • Definition 1.2
  • Theorem 1.3: Uniform sampling from warm start
  • Theorem 1.4: Restricted Gaussian sampling from warm start
  • Theorem 1.5
  • Theorem 1.6
  • Corollary 1.7
  • Theorem 1.8: Uniform sampling from cold start
  • Corollary 1.9: Restricted Gaussian sampling from cold start
  • Theorem 1.10: Logconcave sampling from cold start
  • ...and 75 more