Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Total Variation Guarantees for Sampling with Stochastic Localization

Jakob Kellermann

Abstract

Motivated by the success of score-based generative models, a number of diffusion-based algorithms have recently been proposed for the problem of sampling from a probability measure whose unnormalized density can be accessed. Among them, Grenioux et al. introduced SLIPS, a sampling algorithm based on Stochastic Localization. While SLIPS exhibits strong empirical performance, no rigorous convergence analysis has previously been provided. In this work, we close this gap by establishing the first guarantee for SLIPS in total variation distance. Under minimal assumptions on the target, our bound implies that the number of steps required to achieve an $\varepsilon$-guarantee scales linearly with the dimension, up to logarithmic factors. The analysis leverages techniques from the theory of score-based generative models and further provides theoretical insights into the empirically observed optimal choice of discretization points.

Total Variation Guarantees for Sampling with Stochastic Localization

Abstract

Motivated by the success of score-based generative models, a number of diffusion-based algorithms have recently been proposed for the problem of sampling from a probability measure whose unnormalized density can be accessed. Among them, Grenioux et al. introduced SLIPS, a sampling algorithm based on Stochastic Localization. While SLIPS exhibits strong empirical performance, no rigorous convergence analysis has previously been provided. In this work, we close this gap by establishing the first guarantee for SLIPS in total variation distance. Under minimal assumptions on the target, our bound implies that the number of steps required to achieve an -guarantee scales linearly with the dimension, up to logarithmic factors. The analysis leverages techniques from the theory of score-based generative models and further provides theoretical insights into the empirically observed optimal choice of discretization points.

Paper Structure

This paper contains 21 sections, 15 theorems, 105 equations, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Outline.
  4. Notation.
  5. SLIPS sampling algorithm
  6. Stochastic Localization
  7. SLIPS: The standard case
  8. SLIPS: General denoising schedules
  9. Stochastic Localization and SGM
  10. Sampling guarantees in TV distance
  11. Main results
  12. Proof strategy
  13. Optimal choice of discretization
  14. SLIPS vs. RDMC
  15. Conclusion
  16. ...and 6 more sections

Key Result

Theorem 3.2

Assume Assumptions assumption_moment and assumption_posterior_estimator_SL_process, let $K\in \mathbb{N}$, $0<t_0<t_K$, and write $T=t_K$. Consider an arbitrary discretization $(t_k)_{0\leq k \leq K}$ of $[t_0,T]$. Then, the law $\tilde{\pi}_{T} \coloneqq (1/T)_\# \tilde{p}_T$ of the output of SLIPS where $\varepsilon_0$ is from Assumption assumption_posterior_estimator_SL_process and

Theorems & Definitions (39)

  • Theorem 3.2
  • Corollary 3.3: Informal
  • Remark 3.4
  • Remark 3.5
  • Lemma 3.6
  • Lemma 3.7
  • Remark 3.8
  • Lemma 4.1
  • Remark 4.2
  • Remark 4.3
  • ...and 29 more