HRGR: Enhancing Image Manipulation Detection via Hierarchical Region-aware Graph Reasoning

TL;DR

This work tackles image manipulation detection by replacing grid-based correlation modeling with Hierarchical Region-aware Graph Reasoning (HRGR). HRGR constructs content-coherent irregular regions via Differentiable Feature Partition, builds a hierarchical graph that spans intra- and inter-layer regions, and applies structure-agnostic graph reasoning to refine node representations in an end-to-end manner. The approach yields improvements over state-of-the-art methods across multiple datasets with minimal additional parameters and can serve as a versatile plug-in for existing detectors. The results underscore HRGR's ability to better capture manipulation traces by aligning regional features across scales and layers, enhancing localization accuracy and robustness to perturbations.

Abstract

Image manipulation detection is to identify the authenticity of each pixel in images. One typical approach to uncover manipulation traces is to model image correlations. The previous methods commonly adopt the grids, which are fixed-size squares, as graph nodes to model correlations. However, these grids, being independent of image content, struggle to retain local content coherence, resulting in imprecise detection.To address this issue, we describe a new method named Hierarchical Region-aware Graph Reasoning (HRGR) to enhance image manipulation detection. Unlike existing grid-based methods, we model image correlations based on content-coherence feature regions with irregular shapes, generated by a novel Differentiable Feature Partition strategy. Then we construct a Hierarchical Region-aware Graph based on these regions within and across different feature layers. Subsequently, we describe a structural-agnostic graph reasoning strategy tailored for our graph to enhance the representation of nodes. Our method is fully differentiable and can seamlessly integrate into mainstream networks in an end-to-end manner, without requiring additional supervision. Extensive experiments demonstrate the effectiveness of our method in image manipulation detection, exhibiting its great potential as a plug-and-play component for existing architectures. Codes and models are available at https://github.com/OUC-VAS/HRGR-IMD.

Figures (5)

• Figure 1: The difference between (a) previous methods and (b) ours. Instead of using regular grids in previous methods, we model image correlations across different scales (orange and blue arrows) using connected local content-coherent feature regions.
• Figure 2: Overview of our method. We first extract the feature regions on feature layers from the encoder using a Differentiable Feature Partition (DFP) strategy. Then we build the hierarchical region-aware graph over all features and perform graph reasoning to update the features. The process of Differentiable Feature Partition and Hierarchical Graph Reasoning and Reasoning are shown in the right part.
• Figure 3: Illustration of adjacent matrix $\mathbf{A}$. We build a hierarchical graph that establishes adjacency relationships within the same layer (intra-layer, orange arrows in Fig. \ref{['fig:concept']} (b)) and also across different layers (inter-layer, blue arrows in Fig. \ref{['fig:concept']} (b)), resulting in a single adjacent matrix $\mathbf{A}$. For simplicity, we use an example with two layers (highlighted in blue with one node and green with four nodes, see Fig \ref{['fig:relationships']} (a)). First, we resample them to the same resolution (see Fig \ref{['fig:relationships']} (b)). Then we stack them along channel dimension and apply the 3D sliding window strategy to obtain an adjacency matrix (see Fig \ref{['fig:relationships']} (c,d)).
• Figure 4: Qualitative visualization on CASIA, IMD20, NIST16, Coverage, and Columbia datasets (from top to bottom).
• Figure 5: Robustness analysis under various perturbations on CASIA dataset.