Towards Exploring Fairness in Visual Transformer based Natural and GAN Image Detection Systems

TL;DR

This work investigates algorithmic fairness in visual-transformer based image-forensic systems that distinguish natural from GAN-generated images. It introduces a two-phase bias evaluation framework (uncompressed and JPEG-compressed data) and assesses ViT, CvT, and Swin across gender, race, affective, and intersectional domains. Uncompressed results show bias primarily in ViT, while CvT and Swin appear fair; compression, however, amplifies biases across all models, especially for GAN predictions. The study provides a practical bias-evaluation blueprint and highlights the need for fairness-aware design and mitigation in forensic systems, with materials to be publicly released for replication and extension.

Abstract

Image forensics research has recently witnessed a lot of advancements towards developing computational models capable of accurately detecting natural images captured by cameras and GAN generated images. However, it is also important to ensure whether these computational models are fair enough and do not produce biased outcomes that could eventually harm certain societal groups or cause serious security threats. Exploring fairness in image forensic algorithms is an initial step towards mitigating these biases. This study explores bias in visual transformer based image forensic algorithms that classify natural and GAN images, since visual transformers are recently being widely used in image classification based tasks, including in the area of image forensics. The proposed study procures bias evaluation corpora to analyze bias in gender, racial, affective, and intersectional domains using a wide set of individual and pairwise bias evaluation measures. Since the robustness of the algorithms against image compression is an important factor to be considered in forensic tasks, this study also analyzes the impact of image compression on model bias. Hence to study the impact of image compression on model bias, a two-phase evaluation setting is followed, where the experiments are carried out in uncompressed and compressed evaluation settings. The study could identify bias existences in the visual transformer based models distinguishing natural and GAN images, and also observes that image compression impacts model biases, predominantly amplifying the presence of biases in class GAN predictions.

Figures (4)

• Figure 1: The overall architecture of the proposed work. The visual transformer based forensic classifier system is elaborated in figure \ref{['fig_3bias_model']}
• Figure 2: Visual transformer based forensic classifier system
• Figure 3: Prediction intensity plots of an unbiased intersectional pair Dark skin Male vs. Light skin Male (D+M $\times$ L+M) in the bias evaluation corpora
• Figure 4: Prediction intensity plots of a biased intersectional pair Light skin Female vs. Dark skin Male (L+F $\times$ D+M) in the bias evaluation corpora