CBDiff:Conditional Bernoulli Diffusion Models for Image Forgery Localization

TL;DR

This work tackles image forgery localization under uncertainty by moving beyond a single deterministic mask to generate multiple plausible localization maps. It introduces CBDiff, a conditional Bernoulli diffusion approach that uses Bernoulli noise and a Time-Step Cross-Attention (TSCAttention) module, conditioning on the forged image, Noiseprint++ features, and multi-scale semantic cues from a DINO backbone to produce diverse tampering masks. The method optimizes a variational objective with a simplified KL term due to the Bernoulli forward and yields eight predictions per test sample whose confidence is aggregated via voting, achieving state-of-the-art performance and robustness across eight datasets. The results demonstrate a practical, uncertainty-aware paradigm for forensic localization with potential for more efficient, edge-friendly deployments and multimodal extensions.

Abstract

Image Forgery Localization (IFL) is a crucial task in image forensics, aimed at accurately identifying manipulated or tampered regions within an image at the pixel level. Existing methods typically generate a single deterministic localization map, which often lacks the precision and reliability required for high-stakes applications such as forensic analysis and security surveillance. To enhance the credibility of predictions and mitigate the risk of errors, we introduce an advanced Conditional Bernoulli Diffusion Model (CBDiff). Given a forged image, CBDiff generates multiple diverse and plausible localization maps, thereby offering a richer and more comprehensive representation of the forgery distribution. This approach addresses the uncertainty and variability inherent in tampered regions. Furthermore, CBDiff innovatively incorporates Bernoulli noise into the diffusion process to more faithfully reflect the inherent binary and sparse properties of forgery masks. Additionally, CBDiff introduces a Time-Step Cross-Attention (TSCAttention), which is specifically designed to leverage semantic feature guidance with temporal steps to improve manipulation detection. Extensive experiments on eight publicly benchmark datasets demonstrate that CBDiff significantly outperforms existing state-of-the-art methods, highlighting its strong potential for real-world deployment.

Figures (4)

• Figure 1: Comparison between the proposed method (CBDiff) and the previous approaches (MPC MPC and DiffForensics DiffForensics) in terms of forgery mask generation. Existing approaches output a single deterministic mask, limiting its robustness to uncertain cases. In contrast, CBDiff produces multiple candidate masks, capturing uncertainty and enhancing interpretability. For simple forgeries, CBDiff outputs are almost consistent; for complex cases, its predictions show greater diversity and coverage.
• Figure 2: Overall architecture of CBDiff framework. Given an input image $Y$, semantic features $F$ are extracted via a DINO-pretrained backbone caron2021emerging, while noise residuals $N$ are computed using a Noiseprint++ extractor Trufor. These are fused as conditional inputs to guide the denoising process of the Bernoulli-based diffusion model (DM). Starting from pure Bernoulli noise $X_T$, the model progressively refines predictions across $T$ iterations to generate diverse candidate masks $\hat{X}_0$. The TSAttention module injects multi-scale semantic features $f_i$ into the diffusion process, enhancing forgery localization accuracy and robustness.
• Figure 3: Architecture of Time-Step Cross-Attention (TSCAttention) module. Input features are linearly projected to obtain the query $Q$, while modulated DINO features serve as keys $K$ and values $V$ after linear transformation. A learnable scale-and-shift operation, modulated by time embeddings, is applied to the DINO features. The resulting attention map is computed via scaled dot-product attention and passed through a linear projection and dropout layer. This design enables effective injection of semantic and temporal information into the diffusion process.
• Figure 4: Qualitative comparison of forgery localization methods on example testing images. From left to right: two copy-move images, two splicing images, two removal images and two local AI-generated images. From top to bottom: tampered image, ground truth (GT), and the localization results from CAT-Net, TruFor, MPC and CBDiff.