Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cryptographic Hardness of Score Estimation

Min Jae Song

TL;DR

This work shows that, absent strong data-distribution assumptions, achieving $L^2$-accurate score estimation is computationally hard even with polynomial sample complexity. By a reduction from the Gaussian pancakes problem—cryptographically hard-to-distinguish from the Gaussian under lattice-based assumptions—the authors establish a statistical-to-computational gap for score estimation in diffusion models. The key mechanism is a forward-process score-based reduction that, given an $L^2$-accurate score oracle with error $O(1/\sqrt{\log d})$, would solve pancake-distinguishing, contradicting cryptographic hardness in regimes with $\\gamma\\sigma=O(1)$. The results imply that efficient score estimation requires stronger distributional assumptions than those captured by Gaussian pancakes, with implications for diffusion-model sampling and the design of feasible score-estimation pipelines. Overall, the paper connects diffusion theory, score estimation, and cryptographic hardness to delineate fundamental limits on learning-based score estimation.

Abstract

We show that $L^2$-accurate score estimation, in the absence of strong assumptions on the data distribution, is computationally hard even when sample complexity is polynomial in the relevant problem parameters. Our reduction builds on the result of Chen et al. (ICLR 2023), who showed that the problem of generating samples from an unknown data distribution reduces to $L^2$-accurate score estimation. Our hard-to-estimate distributions are the "Gaussian pancakes" distributions, originally due to Diakonikolas et al. (FOCS 2017), which have been shown to be computationally indistinguishable from the standard Gaussian under widely believed hardness assumptions from lattice-based cryptography (Bruna et al., STOC 2021; Gupte et al., FOCS 2022).

Cryptographic Hardness of Score Estimation

TL;DR

This work shows that, absent strong data-distribution assumptions, achieving -accurate score estimation is computationally hard even with polynomial sample complexity. By a reduction from the Gaussian pancakes problem—cryptographically hard-to-distinguish from the Gaussian under lattice-based assumptions—the authors establish a statistical-to-computational gap for score estimation in diffusion models. The key mechanism is a forward-process score-based reduction that, given an -accurate score oracle with error , would solve pancake-distinguishing, contradicting cryptographic hardness in regimes with . The results imply that efficient score estimation requires stronger distributional assumptions than those captured by Gaussian pancakes, with implications for diffusion-model sampling and the design of feasible score-estimation pipelines. Overall, the paper connects diffusion theory, score estimation, and cryptographic hardness to delineate fundamental limits on learning-based score estimation.

Abstract

We show that -accurate score estimation, in the absence of strong assumptions on the data distribution, is computationally hard even when sample complexity is polynomial in the relevant problem parameters. Our reduction builds on the result of Chen et al. (ICLR 2023), who showed that the problem of generating samples from an unknown data distribution reduces to -accurate score estimation. Our hard-to-estimate distributions are the "Gaussian pancakes" distributions, originally due to Diakonikolas et al. (FOCS 2017), which have been shown to be computationally indistinguishable from the standard Gaussian under widely believed hardness assumptions from lattice-based cryptography (Bruna et al., STOC 2021; Gupte et al., FOCS 2022).
Paper Structure (27 sections, 19 theorems, 112 equations, 1 figure)

This paper contains 27 sections, 19 theorems, 112 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Main contributions
  3. Related work
  4. Theory of diffusion models.
  5. Score estimation.
  6. Statistical-to-computational gaps.
  7. Gaussian pancakes.
  8. Future directions
  9. Preliminaries
  10. Notation.
  11. Background on denoising diffusion probabilistic modeling (DDPM)
  12. Lattices and discrete Gaussians
  13. Lattices.
  14. Fourier analysis.
  15. Discrete Gaussians.
  16. ...and 12 more sections

Key Result

Theorem 1.1

Let $\gamma(d) > 1, \sigma(d) > 0$ be sequences such that $\sigma \ge 1/\mathrm{poly}(d)$ and the corresponding $(\gamma,\sigma)$-Gaussian pancakes distributions $(P_{\bm u})_{\bm u \in \mathbb{S}^{d-1}}$ all satisfy $\mathrm{TV}(P_{\bm u}, \mathcal{N}(0,I_d)) > 1/2$. Then, there exists a polynomial

Figures (1)

  • Figure 1: Top: Scatter plot of 2D Gaussian pancakes $P_{\bm u}$ with secret direction $\bm u = (-1/\sqrt{2},1/\sqrt{2})$, spacing $\gamma=6$, and thickness $\sigma \in \{0.01, 0.05, 0.25\}$. Bottom: Re-scaled probability densities of Gaussian pancakes (blue) for each $\sigma \in \{0.01, 0.05, 0.25\}$ and the standard Gaussian (black) along $\bm u$. For fixed $\gamma$, the pancakes "blur into each other" as $\sigma$ increases.

Theorems & Definitions (48)

  • Theorem 1.1: Informal, see Theorem \ref{['thm:reduction-to-score-estimation']}
  • Lemma 2.1: Poisson summation formula
  • Definition 2.2: Gaussian function
  • Remark 2.3: Non-standard choice of "standard" variance
  • Definition 2.4: Discrete Gaussian
  • Definition 2.5: Smoothed discrete Gaussian
  • Claim 2.6: Smoothed discrete Gaussian density
  • proof
  • Remark 2.7: Periodic Gaussian
  • Definition 2.8: Gaussian pancakes
  • ...and 38 more