Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On theoretical guarantees and a blessing of dimensionality for nonconvex sampling

Martin Chak

TL;DR

The paper develops explicit, scalable guarantees for sampling from high-dimensional nonconvex targets by introducing a tail-matching flattened proposal $\pi$ and correcting with importance sampling. It proves a polynomial-time sampling regime when the nonconvexity radius $R$ scales no faster than $\sqrt{d}$, and provides precise bounds on the chi-square divergence between the target and the proposal, controlled by Gaussian-tail assumptions. Conversely, it establishes sharp intractability results: under strong convexity outside a ball or mere dissipativity, there exist targets for which any algorithm requires exponential work in dimension to achieve accurate estimates. The results are illustrated with Gaussian mixtures and Bayesian neural networks, showing how the framework yields tractable sampling under concrete architectural constraints and tail behavior, thus bridging theory and scalable practice in nonconvex sampling.

Abstract

Existing guarantees for algorithms sampling from nonlogconcave measures on $\mathbb{R}^d$ are generally inexplicit or unscalable. Even for the class of measures with logdensities that have bounded Hessians and are strongly concave outside a Euclidean ball of radius $R$, no available theory is comprehensively satisfactory with respect to both $R$ and $d$. In this paper, it is shown that complete polynomial complexity can in fact be achieved if $R\leq c\sqrt{d}$, whilst an exponential number of point evaluations is generally necessary for any algorithm as soon as $R\geq C\sqrt{d}$ for constants $C>c>0$. Importance sampling with a tail-matching proposal achieves the former, owing to a blessing of dimensionality. On the other hand, if strong concavity outside a ball is replaced by a distant dissipativity condition, then sampling guarantees must generally scale exponentially with $d$ in all parameter regimes.

On theoretical guarantees and a blessing of dimensionality for nonconvex sampling

TL;DR

The paper develops explicit, scalable guarantees for sampling from high-dimensional nonconvex targets by introducing a tail-matching flattened proposal and correcting with importance sampling. It proves a polynomial-time sampling regime when the nonconvexity radius scales no faster than , and provides precise bounds on the chi-square divergence between the target and the proposal, controlled by Gaussian-tail assumptions. Conversely, it establishes sharp intractability results: under strong convexity outside a ball or mere dissipativity, there exist targets for which any algorithm requires exponential work in dimension to achieve accurate estimates. The results are illustrated with Gaussian mixtures and Bayesian neural networks, showing how the framework yields tractable sampling under concrete architectural constraints and tail behavior, thus bridging theory and scalable practice in nonconvex sampling.

Abstract

Existing guarantees for algorithms sampling from nonlogconcave measures on are generally inexplicit or unscalable. Even for the class of measures with logdensities that have bounded Hessians and are strongly concave outside a Euclidean ball of radius , no available theory is comprehensively satisfactory with respect to both and . In this paper, it is shown that complete polynomial complexity can in fact be achieved if , whilst an exponential number of point evaluations is generally necessary for any algorithm as soon as for constants . Importance sampling with a tail-matching proposal achieves the former, owing to a blessing of dimensionality. On the other hand, if strong concavity outside a ball is replaced by a distant dissipativity condition, then sampling guarantees must generally scale exponentially with in all parameter regimes.

Paper Structure

This paper contains 16 sections, 20 theorems, 146 equations, 1 algorithm.

Table of Contents

  1. Introduction
  2. Summary of results
  3. Notation
  4. Chi-square divergence estimate
  5. Main estimates
  6. Application to known assumptions
  7. Intractability
  8. Strong convexity outside a ball
  9. Dissipativity
  10. Sampling guarantees for the flattened distribution
  11. Examples
  12. Gaussian mixtures
  13. Bayesian neural networks
  14. Mathematical setting
  15. Flexibility and scope
  16. ...and 1 more sections

Key Result

Theorem 2.1

Let $c\in(0,\infty)$, $M\in[\min_{\mathbb{R}^d}U,\infty)$ and let $\pi$ be a probability distribution with density given by pidef for some smooth nondecreasing $T:[0,\infty)\rightarrow[0,\infty)$ satisfying Tdef and $T"\geq 0$. Let $(u^n)_{n\in\mathbb{N}}$ be an i.i.d. sequence of $\mathbb{R}^d$-val where the supremums are over all measurable $\phi:\mathbb{R}^d\rightarrow\mathbb{R}$ with $\sup_x\l

Theorems & Definitions (45)

  • Theorem 2.1
  • Lemma 2.2
  • proof : Proof of Lemma \ref{['base']}
  • Theorem 2.3
  • proof
  • Corollary 2.4
  • Remark 2.1
  • proof : Proof of Corollary \ref{['coco2']}
  • Proposition 2.5
  • proof
  • ...and 35 more