Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Image Forgery Localization with State Space Models

Zijie Lou, Gang Cao, Kun Guo, Shaowei Weng, Lifang Yu

TL;DR

LoMa addresses image forgery localization by modeling pixel dependencies with selective state space models (SSMs). It combines global dependency modeling via Mixed-SSM Blocks with an atrous selective scan and local CNN-based processing, culminating in a lightweight decoder to produce pixel-wise forgery maps, all with linear computational complexity. Across 10 public datasets, LoMa surpasses CNN- and Transformer-based state-of-the-art methods while reducing computation, demonstrating strong generalization and robustness to OSN post-processing. This work establishes a baseline for SSM-based forgery localization and points to future efficient SSM-driven forensic models.

Abstract

Pixel dependency modeling from tampered images is pivotal for image forgery localization. Current approaches predominantly rely on Convolutional Neural Networks (CNNs) or Transformer-based models, which often either lack sufficient receptive fields or entail significant computational overheads. Recently, State Space Models (SSMs), exemplified by Mamba, have emerged as a promising approach. They not only excel in modeling long-range interactions but also maintain a linear computational complexity. In this paper, we propose LoMa, a novel image forgery localization method that leverages the selective SSMs. Specifically, LoMa initially employs atrous selective scan to traverse the spatial domain and convert the tampered image into ordered patch sequences, and subsequently applies multi-directional state space modeling. In addition, an auxiliary convolutional branch is introduced to enhance local feature extraction. Extensive experimental results validate the superiority of LoMa over CNN-based and Transformer-based state-of-the-arts. To our best knowledge, this is the first image forgery localization model constructed based on the SSM-based model. We aim to establish a baseline and provide valuable insights for the future development of more efficient and effective SSM-based forgery localization models. Code is available at https://github.com/multimediaFor/LoMa.

Image Forgery Localization with State Space Models

TL;DR

LoMa addresses image forgery localization by modeling pixel dependencies with selective state space models (SSMs). It combines global dependency modeling via Mixed-SSM Blocks with an atrous selective scan and local CNN-based processing, culminating in a lightweight decoder to produce pixel-wise forgery maps, all with linear computational complexity. Across 10 public datasets, LoMa surpasses CNN- and Transformer-based state-of-the-art methods while reducing computation, demonstrating strong generalization and robustness to OSN post-processing. This work establishes a baseline for SSM-based forgery localization and points to future efficient SSM-driven forensic models.

Abstract

Pixel dependency modeling from tampered images is pivotal for image forgery localization. Current approaches predominantly rely on Convolutional Neural Networks (CNNs) or Transformer-based models, which often either lack sufficient receptive fields or entail significant computational overheads. Recently, State Space Models (SSMs), exemplified by Mamba, have emerged as a promising approach. They not only excel in modeling long-range interactions but also maintain a linear computational complexity. In this paper, we propose LoMa, a novel image forgery localization method that leverages the selective SSMs. Specifically, LoMa initially employs atrous selective scan to traverse the spatial domain and convert the tampered image into ordered patch sequences, and subsequently applies multi-directional state space modeling. In addition, an auxiliary convolutional branch is introduced to enhance local feature extraction. Extensive experimental results validate the superiority of LoMa over CNN-based and Transformer-based state-of-the-arts. To our best knowledge, this is the first image forgery localization model constructed based on the SSM-based model. We aim to establish a baseline and provide valuable insights for the future development of more efficient and effective SSM-based forgery localization models. Code is available at https://github.com/multimediaFor/LoMa.

Paper Structure

This paper contains 16 sections, 6 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Proposed LoMa Scheme
  3. Preliminaries
  4. Overall Pipeline
  5. Efficient SS2D
  6. Experiments
  7. Experimental Settings
  8. Datasets
  9. Compared Methods
  10. Evaluation metrics
  11. Implementation details
  12. Loss function
  13. Comparison with State-of-the-art Methods
  14. Robustness Evaluation
  15. Ablation Studies
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Proposed image forgery localization scheme LoMa. $N_i$={2, 2, 9, 2}, LN, BN and CAB refer to Layer Normalization, Batch Normalization, and Channel Attention Block, respectively.
  • Figure 2: Illustration of efficient 2D-Selective-Scan (SS2D).
  • Figure 3: Qualitative comparison of forgery localization methods on example testing images. From left to right: two splicing images, two copy-move images, two object-removal images, two AI edited images and two deepfake images. From top to bottom: tampered image, ground truth (GT), and the localization results from the best CNN-based method CAT-Net, the best Transformer-based method MPC and LoMa.