Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TwoHead-SwinFPN: A Unified DL Architecture for Synthetic Manipulation, Detection and Localization in Identity Documents

Chan Naseeb, Adeel Ashraf Cheema, Hassan Sami, Tayyab Afzal, Muhammad Omair, Usman Habib

TL;DR

The paper tackles AI-driven manipulations in identity documents by introducing TwoHead-SwinFPN, a unified architecture that performs both detection and precise localization of manipulations. It combines a Swin Transformer backbone with FPN for multi-scale feature fusion and a CBAM-enhanced UNet-style decoder, guarded by uncertainty-weighted multi-task learning to jointly optimize classification and segmentation. On the FantasyIDiap dataset, the approach achieves 84.31% accuracy, 90.78% AUC, and a mean Dice score of 57.24% for localization, while enabling sub-second deployment via a FastAPI service. The work is supported by thorough ablations, cross-device and cross-language analyses, and a plan for future enhancements including frequency-domain analysis and adversarial robustness to sustain performance against evolving manipulation techniques.

Abstract

The proliferation of sophisticated generative AI models has significantly escalated the threat of synthetic manipulations in identity documents, particularly through face swapping and text inpainting attacks. This paper presents TwoHead-SwinFPN, a unified deep learning architecture that simultaneously performs binary classification and precise localization of manipulated regions in ID documents. Our approach integrates a Swin Transformer backbone with Feature Pyramid Network (FPN) and UNet-style decoder, enhanced with Convolutional Block Attention Module (CBAM) for improved feature representation. The model employs a dual-head architecture for joint optimization of detection and segmentation tasks, utilizing uncertainty-weighted multi-task learning. Extensive experiments on the FantasyIDiap dataset demonstrate superior performance with 84.31\% accuracy, 90.78\% AUC for classification, and 57.24\% mean Dice score for localization. The proposed method achieves an F1-score of 88.61\% for binary classification while maintaining computational efficiency suitable for real-world deployment through FastAPI implementation. Our comprehensive evaluation includes ablation studies, cross-device generalization analysis, and detailed performance assessment across 10 languages and 3 acquisition devices.

TwoHead-SwinFPN: A Unified DL Architecture for Synthetic Manipulation, Detection and Localization in Identity Documents

TL;DR

The paper tackles AI-driven manipulations in identity documents by introducing TwoHead-SwinFPN, a unified architecture that performs both detection and precise localization of manipulations. It combines a Swin Transformer backbone with FPN for multi-scale feature fusion and a CBAM-enhanced UNet-style decoder, guarded by uncertainty-weighted multi-task learning to jointly optimize classification and segmentation. On the FantasyIDiap dataset, the approach achieves 84.31% accuracy, 90.78% AUC, and a mean Dice score of 57.24% for localization, while enabling sub-second deployment via a FastAPI service. The work is supported by thorough ablations, cross-device and cross-language analyses, and a plan for future enhancements including frequency-domain analysis and adversarial robustness to sustain performance against evolving manipulation techniques.

Abstract

The proliferation of sophisticated generative AI models has significantly escalated the threat of synthetic manipulations in identity documents, particularly through face swapping and text inpainting attacks. This paper presents TwoHead-SwinFPN, a unified deep learning architecture that simultaneously performs binary classification and precise localization of manipulated regions in ID documents. Our approach integrates a Swin Transformer backbone with Feature Pyramid Network (FPN) and UNet-style decoder, enhanced with Convolutional Block Attention Module (CBAM) for improved feature representation. The model employs a dual-head architecture for joint optimization of detection and segmentation tasks, utilizing uncertainty-weighted multi-task learning. Extensive experiments on the FantasyIDiap dataset demonstrate superior performance with 84.31\% accuracy, 90.78\% AUC for classification, and 57.24\% mean Dice score for localization. The proposed method achieves an F1-score of 88.61\% for binary classification while maintaining computational efficiency suitable for real-world deployment through FastAPI implementation. Our comprehensive evaluation includes ablation studies, cross-device generalization analysis, and detailed performance assessment across 10 languages and 3 acquisition devices.
Paper Structure (46 sections, 10 equations, 3 figures, 9 tables, 1 algorithm)

This paper contains 46 sections, 10 equations, 3 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contemporary Forgery Detection Methods
  4. Deep Learning in Media Forensics
  5. Face Swapping Detection
  6. Text Inpainting and Localization
  7. Multi-task Learning in Forensics
  8. Dataset Analysis
  9. FantasyIDiap Dataset Overview
  10. Language Distribution Analysis
  11. Device-Specific Characteristics
  12. Manipulation Analysis
  13. Quality Assessment
  14. Methodology
  15. Architecture Overview
  16. ...and 31 more sections

Figures (3)

  • Figure 1: Architecture comparison between baseline methods and our proposed TwoHead-SwinFPN approach.
  • Figure 2: Language distribution in the FantasyIDiap dataset showing balanced representation across 10 languages.
  • Figure 3: Cross-device performance analysis showing consistent results across acquisition devices.