Mitigating Hallucinations in Diffusion Models through Adaptive Attention Modulation

TL;DR

The paper tackles hallucinations in unconditional diffusion models by introducing Adaptive Attention Modulation (AAM), which dynamically tunes the self-attention sharpness during denoising and supplements it with masked perturbations to suppress early anomalous regions. By leveraging an inference-time optimization guided by a PatchCore anomaly signal and a memory bank built from the denoising UNet, the approach achieves robust reductions in hallucinations and improvements in FID across multiple datasets. Key contributions include identifying the critical role of early denoising attention, proposing a practical adaptive temperature framework, and validating the method with extensive ablations that demonstrate additive gains from each component. The proposed technique offers a practical route to more faithful unconditional diffusion outputs with improved reliability for downstream applications.

Abstract

Diffusion models, while increasingly adept at generating realistic images, are notably hindered by hallucinations -- unrealistic or incorrect features inconsistent with the trained data distribution. In this work, we propose Adaptive Attention Modulation (AAM), a novel approach to mitigate hallucinations by analyzing and modulating the self-attention mechanism in diffusion models. We hypothesize that self-attention during early denoising steps may inadvertently amplify or suppress features, contributing to hallucinations. To counter this, AAM introduces a temperature scaling mechanism within the softmax operation of the self-attention layers, dynamically modulating the attention distribution during inference. Additionally, AAM employs a masked perturbation technique to disrupt early-stage noise that may otherwise propagate into later stages as hallucinations. Extensive experiments demonstrate that AAM effectively reduces hallucinatory artifacts, enhancing both the fidelity and reliability of generated images. For instance, the proposed approach improves the FID score by 20.8% and reduces the percentage of hallucinated images by 12.9% (in absolute terms) on the Hands dataset.

Figures (5)

• Figure 1: Overview of our proposed pipeline for mitigating hallucinations in diffusion models with adaptive temperature scaling. At each denoising timestep $t$, the noisy input $x_t$ is processed through a denoising UNet comprising self-attention layers. Conventional self-attention (top-right) employs a fixed attention distribution, which may inadvertently amplify or suppress features, leading to hallucinations in the generated image $x_0$. In contrast, our approach (bottom-right) introduces a temperature-scaled self-attention, where the temperature $\tau$, is dynamically adjusted based on an anomaly score $s$ from an anomaly detection model. This adaptive mechanism modulates the attention distribution, reducing noise-induced artifacts while preserving essential features, thereby enhancing the overall realism and fidelity of the generated images.
• Figure 2: Examples of affirmative and detrimental attention behaviors in diffusion models. The columns $\hat{x}_0^{(t_1)}$ and $\hat{x}_0^{(t_1-\Delta_t)}$ show the intermediate denoised predictions at an early denoising step $t_1$ and a subsequent step $t_1 - \Delta_t$. The $x_0$ column depicts the final denoised image. Rows 1 and 3 illustrate affirmative emphasis and suppression, respectively, resulting in realistic denoised images, while rows 2 and 4 show detrimental emphasis and suppression, leading to hallucinations in the generated images.
• Figure 3: Example generations with temperature scaled attention. For each column, we start with the same initial noise ($x_T$), which results in hallucinations under the default setting with no temperature scaling (i.e.$\tau=1.0$). Each row depicts the generated samples for different combinations of $\tau_{32}$ and $\tau_{16}$, corresponding to the resolutions at which the self-attention layers operate --- $32\times32$ and $16\times 16$, respectively. Hallucination-free generations are indicated with green borders. This figure demonstrates that adjusting $\tau$ can mitigate hallucinations, although no single temperature value provides a universal solution across all scenarios.
• Figure 4: Effectiveness of hallucination mitigation across datasets. Visual comparison of generated samples with (green border) and without (red border) our proposed method. From left to right: (a) Simple Shapes dataset aithal2024understanding, where our method prevents the generation of extra instances, adhering to the single-instance-per-region distribution; (b) MNIST dataset lecun1998gradient, where hallucinated artifacts that distort the digits are reduced; (c) Hands dataset afifi201911k, where hallucinations such as missing, extra, or malformed fingers are corrected, producing anatomically accurate hands. Our approach consistently enhances realism and fidelity in generated images across different datasets.
• Figure 5: Effect of adaptive temperature tuning and masked perturbation. The default diffusion model (left) commonly generates artifacts like missing or extra fingers. Adaptive temperature tuning (middle) reduces such artifacts to a certain extent, while combining it with masked perturbation (right) effectively mitigates hallucinations, producing anatomically accurate hands.