Distributional Shrinkage I: Universal Denoisers in Multi-Dimensions

Tengyuan Liang

Paper

Distributional Shrinkage I: Universal Denoisers in Multi-Dimensions

Abstract

We revisit the problem of denoising from noisy measurements where only the noise level is known, not the noise distribution. In multi-dimensions, independent noise

corrupts the signal

, resulting in the noisy measurement

, where

is a known noise level. Our goal is to recover the underlying signal distribution

from denoising

. We propose and analyze universal denoisers that are agnostic to a wide range of signal and noise distributions. Our distributional denoisers offer order-of-magnitude improvements over the Bayes-optimal denoiser derived from Tweedie's formula, if the focus is on the entire distribution

rather than on individual realizations of

. Our denoisers shrink

toward

optimally, achieving

and

accuracy in matching generalized moments and density functions. Inspired by optimal transport theory, the proposed denoisers are optimal in approximating the Monge-Ampère equation with higher-order accuracy, and can be implemented efficiently via score matching. Let

represent the density of

; for optimal distributional denoising, we recommend replacing the Bayes-optimal denoiser,

with denoisers exhibiting less aggressive distributional shrinkage,