Squeezed Diffusion Models
Jyotirmai Singh, Samar Khanna, James Burgess
TL;DR
Diffusion models typically rely on isotropic noise, which can mask structure in data with anisotropic content. We propose Squeezed Diffusion Models (SDM), applying a PCA-aligned, time-varying anisotropic noise scale inspired by quantum squeezing, and study two variants: Heisenberg diffusion (HDM) and standard SDM. A key finding is that mild antisqueezing along the principal axis yields meaningful gains, with up to $\sim 15\%$ improvement in FID on CIFAR-10/100 and CelebA-64, while shifting the precision–recall frontier toward higher recall; these gains come without architectural changes. This work demonstrates that data-aware noise shaping is a practical and robust way to boost diffusion model quality, motivating extensions to higher-resolution data and frequency-specific squeezing in future research.
Abstract
Diffusion models typically inject isotropic Gaussian noise, disregarding structure in the data. Motivated by the way quantum squeezed states redistribute uncertainty according to the Heisenberg uncertainty principle, we introduce Squeezed Diffusion Models (SDM), which scale noise anisotropically along the principal component of the training distribution. As squeezing enhances the signal-to-noise ratio in physics, we hypothesize that scaling noise in a data-dependent manner can better assist diffusion models in learning important data features. We study two configurations: (i) a Heisenberg diffusion model that compensates the scaling on the principal axis with inverse scaling on orthogonal directions and (ii) a standard SDM variant that scales only the principal axis. Counterintuitively, on CIFAR-10/100 and CelebA-64, mild antisqueezing - i.e. increasing variance on the principal axis - consistently improves FID by up to 15% and shifts the precision-recall frontier toward higher recall. Our results demonstrate that simple, data-aware noise shaping can deliver robust generative gains without architectural changes.