Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TGIF: Text-Guided Inpainting Forgery Dataset

Hannes Mareen, Dimitrios Karageorgiou, Glenn Van Wallendael, Peter Lambert, Symeon Papadopoulos

TL;DR

This work demonstrates the inefficiency of state-of-the-art detectors on local manipulations performed by modern generative approaches, and aspires to help with the development of more capable IFL and SID methods.

Abstract

Digital image manipulation has become increasingly accessible and realistic with the advent of generative AI technologies. Recent developments allow for text-guided inpainting, making sophisticated image edits possible with minimal effort. This poses new challenges for digital media forensics. For example, diffusion model-based approaches could either splice the inpainted region into the original image, or regenerate the entire image. In the latter case, traditional image forgery localization (IFL) methods typically fail. This paper introduces the Text-Guided Inpainting Forgery (TGIF) dataset, a comprehensive collection of images designed to support the training and evaluation of image forgery localization and synthetic image detection (SID) methods. The TGIF dataset includes approximately 75k forged images, originating from popular open-source and commercial methods, namely SD2, SDXL, and Adobe Firefly. We benchmark several state-of-the-art IFL and SID methods on TGIF. Whereas traditional IFL methods can detect spliced images, they fail to detect regenerated inpainted images. Moreover, traditional SID may detect the regenerated inpainted images to be fake, but cannot localize the inpainted area. Finally, both IFL and SID methods fail when exposed to stronger compression, while they are less robust to modern compression algorithms, such as WEBP. In conclusion, this work demonstrates the inefficiency of state-of-the-art detectors on local manipulations performed by modern generative approaches, and aspires to help with the development of more capable IFL and SID methods. The dataset and code can be downloaded at https://github.com/IDLabMedia/tgif-dataset.

TGIF: Text-Guided Inpainting Forgery Dataset

TL;DR

This work demonstrates the inefficiency of state-of-the-art detectors on local manipulations performed by modern generative approaches, and aspires to help with the development of more capable IFL and SID methods.

Abstract

Digital image manipulation has become increasingly accessible and realistic with the advent of generative AI technologies. Recent developments allow for text-guided inpainting, making sophisticated image edits possible with minimal effort. This poses new challenges for digital media forensics. For example, diffusion model-based approaches could either splice the inpainted region into the original image, or regenerate the entire image. In the latter case, traditional image forgery localization (IFL) methods typically fail. This paper introduces the Text-Guided Inpainting Forgery (TGIF) dataset, a comprehensive collection of images designed to support the training and evaluation of image forgery localization and synthetic image detection (SID) methods. The TGIF dataset includes approximately 75k forged images, originating from popular open-source and commercial methods, namely SD2, SDXL, and Adobe Firefly. We benchmark several state-of-the-art IFL and SID methods on TGIF. Whereas traditional IFL methods can detect spliced images, they fail to detect regenerated inpainted images. Moreover, traditional SID may detect the regenerated inpainted images to be fake, but cannot localize the inpainted area. Finally, both IFL and SID methods fail when exposed to stronger compression, while they are less robust to modern compression algorithms, such as WEBP. In conclusion, this work demonstrates the inefficiency of state-of-the-art detectors on local manipulations performed by modern generative approaches, and aspires to help with the development of more capable IFL and SID methods. The dataset and code can be downloaded at https://github.com/IDLabMedia/tgif-dataset.
Paper Structure (15 sections, 1 figure, 2 tables)

This paper contains 15 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Image Forgery Localization (IFL)
  4. Synthetic Image Detection (SID)
  5. Image Forensics Datasets
  6. TGIF Dataset
  7. Source of Authentic Data
  8. Text-Guided Inpainting Generation
  9. Dataset Splits
  10. Aesthetic Quality and Image-Text-Matching Scores
  11. Benchmark: IFL & SID
  12. Experimental Setup
  13. Image Forgery Localization Methods
  14. Synthetic Image Detection Methods
  15. Discussion & Conclusion

Figures (1)

  • Figure 1: TGIF dataset creation. The original image is fed to SD2, SDXL, and Adobe Firefly (Photoshop), along with a prompt and a mask (segmentation or bounding box). For SD2, an additional splicing step is performed, which is done automatically in Adobe Firefly, and discarded for SDXL's output. For simplicity, only one variation is illustrated, rather than all three variations that we generated in batch.