SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints

TL;DR

SUMI-IFL addresses image forgery localization by grounding feature representation in information-theoretic principles. It introduces sufficiency-view and minimality-view constraints to respectively maximize task-relevant information across multiple forgery views and compress away task-irrelevant details via an information bottleneck-based objective, with a mask-guided encoder–decoder for concise reasoning. The method fuses channel-, spatial-, and pixel-focused forgery cues through a learnable fusion layer and enforces theoretical guarantees with variational bounds and KL regularization. Empirical results across multiple datasets demonstrate state-of-the-art or competitive performance in both in-dataset and cross-dataset settings, with improved robustness to common distortions and clearer manipulation visualizations, underscoring the practical impact of theory-guided IFL.

Abstract

Image forgery localization (IFL) is a crucial technique for preventing tampered image misuse and protecting social safety. However, due to the rapid development of image tampering technologies, extracting more comprehensive and accurate forgery clues remains an urgent challenge. To address these challenges, we introduce a novel information-theoretic IFL framework named SUMI-IFL that imposes sufficiency-view and minimality-view constraints on forgery feature representation. First, grounded in the theoretical analysis of mutual information, the sufficiency-view constraint is enforced on the feature extraction network to ensure that the latent forgery feature contains comprehensive forgery clues. Considering that forgery clues obtained from a single aspect alone may be incomplete, we construct the latent forgery feature by integrating several individual forgery features from multiple perspectives. Second, based on the information bottleneck, the minimality-view constraint is imposed on the feature reasoning network to achieve an accurate and concise forgery feature representation that counters the interference of task-unrelated features. Extensive experiments show the superior performance of SUMI-IFL to existing state-of-the-art methods, not only on in-dataset comparisons but also on cross-dataset comparisons.

Figures (8)

• Figure 1: Illustrate the structure of the feature extraction network. We utilize three backbones to extract the channel forgery feature, the spatial forgery feature, and the pixel forgery feature of the tampered image respectively. Then the three individual image features are fed into the $\mathbf{B}_\phi$ layer to obtain the latent forgery feature. The three backbones are the U-Net structure by substituting the Conv layer with the specially designed attention blocks: channel attention block (CAB), spatial attention block (SAB), and pixel attention block (PAB).
• Figure 2: Overall structure of the proposed SUMI-IFL. The top part is the pipeline, which takes a suspicious image $(H \times W \times 3)$ as input, and the output is the predicted mask $(H \times W \times 1)$. The bottom parts are details of each constraint. The sufficiency-view constraint is applied to the feature extraction network to obtain a latent forgery feature, while the minimality-view constraint is applied to the feature reasoning network to get a concise forgery feature.
• Figure 3: Illustrate the attention blocks.
• Figure 4: Illustrate the structure of the feature reasoning network.
• Figure 5: Robust evaluation against JPEG compression and Gaussian Blurs on DEFACTO.