Energy-Tweedie: Score meets Score, Energy meets Energy

TL;DR

This work extends Tweedie-type identities to a broad class of elliptical (energy-model) noise and establishes an Energy-Score identity that links the Stein score of the noisy marginal to the path-derivative of a matched energy score. By deriving $s_m(y)= -\frac{\lambda}{\beta} \nabla^{PD}_y \mathrm{ES}_{\Sigma^{-1},\beta}(P(X|Y=y), y)$ for generalized Gaussian noise, the authors enable score estimation, parameter calibration, and diffusion-style generation with non-Gaussian, anisotropic, and heavy-tailed noise. The paper provides explicit forms for Gaussian, Laplace, and generalized Gaussian cases, proposes practical estimators from denoising posteriors, and demonstrates a diffusion-like generative framework that broadens the applicability of score-based methods. Collectively, the results offer principled tools for denoising, inverse problems, and generative modeling under diverse noise geometries with potential practical impact on robust and flexible diffusion processes.

Abstract

Denoising and score estimation have long been known to be linked via the classical Tweedie's formula. In this work, we first extend the latter to a wider range of distributions often called "energy models" and denoted elliptical distributions in this work. Next, we examine an alternative view: we consider the denoising posterior $P(X|Y)$ as the optimizer of the energy score (a scoring rule) and derive a fundamental identity that connects the (path-) derivative of a (possibly) non-Euclidean energy score to the score of the noisy marginal. This identity can be seen as an analog of Tweedie's identity for the energy score, and allows for several interesting applications; for example, score estimation, noise distribution parameter estimation, as well as using energy score models in the context of "traditional" diffusion model samplers with a wider array of noising distributions.

Figures (5)

• Figure 1: Diffusion progress from $\sigma=1.0$ to $0.0$. a): "ordinary" Gaussian. b) Generalized Gaussian with $\beta=1.4, \lambda=1.8$. Last image in each row displays the generated samples in orange and the data in blue.
• Figure 2: Mean energy distance to clean data during sampling across 10 different random seeds for a): "ordinary" Gaussian and b) Generalized Gaussian. $\pm$ 1 standard deviation bands are shown in gray, but are barely visible.
• Figure 3: Score fields at $\sigma=0.8$ for a): "ordinary" Gaussian. b) Generalized Gaussian. Noisy data in blue, clean data in red, and samples from the current posterior in green.
• Figure 4: Numeric results for (noisy) score estimation for the Eight Gaussians dataset over 10 random seeds. a): Gaussian, b) Generalized Gaussian. Left: Score MSE, right: Mean cosine similarity.
• Figure 5: Clean data score fields for the Eight Gaussians dataset. a): Gaussian, b) Generalized Gaussian noise. Left: Ground truth field (or importance-sampled one), right: the field obtained via Richardson extrapolation.

Theorems & Definitions (6)

• Definition 3.1: Elliptical distribution
• Proposition 3.2: Tweedie-like formula for elliptical noise
• Lemma 3.3
• Theorem 4.1: The Energy-- Score identity
• Corollary 4.2: Basic Tweedie's Formula is a Consequence
• Corollary 4.3: Ordinary Energy Score Gradient Scales with the Noise Magnitude