Detection of Digital Facial Retouching utilizing Face Beauty Information

TL;DR

The paper investigates detecting facial retouching by leveraging beauty assessment signals as auxiliary information. It combines spatial and frequency features (RGB with face cropping, DCT, SRM) and a Mobile-ViT-based detector, then fuses these signals at the score level, with and without beauty scores from BeholderGAN and MEBeauty. Through careful pre-processing, feature extraction, and both rule-based and ML-based fusion, the approach achieves a state-of-the-art 1.0–1.1% D-EER under a Leave-One-Out setting with unknown retouching algorithms, outperforming prior work. The findings underscore the practical value of beauty information and multi-modal fusion for robust retouching detection in biometric applications, while outlining avenues for expansion to other manipulation types and datasets.

Abstract

Facial retouching to beautify images is widely spread in social media, advertisements, and it is even applied in professional photo studios to let individuals appear younger, remove wrinkles and skin impurities. Generally speaking, this is done to enhance beauty. This is not a problem itself, but when retouched images are used as biometric samples and enrolled in a biometric system, it is one. Since previous work has proven facial retouching to be a challenge for face recognition systems,the detection of facial retouching becomes increasingly necessary. This work proposes to study and analyze changes in beauty assessment algorithms of retouched images, assesses different feature extraction methods based on artificial intelligence in order to improve retouching detection, and evaluates whether face beauty can be exploited to enhance the detection rate. In a scenario where the attacking retouching algorithm is unknown, this work achieved 1.1% D-EER on single image detection.

Figures (7)

• Figure 1: The different face filters used for retouching and the reference bona fide image of the same subject from the Facial Retouch Image dataset Rathgeb2020.
• Figure 2: The KDE-Plots for the FRI-FRGC and FRI-FERET datasets showing the best and worst results of the BeholderGAN versus the MEBeauty classifier.
• Figure 3: The changes between bona fide and retouched filter samples highlighted blue.
• Figure 4: Example of pre-process images and different feature extraction methods are applied. The upper row of images are bona fide images, and the lower rows are retouched by AirBrush. Note that the pixels in the DCT images where highlighted orange for visability.
• Figure 5: The DET curve for the model with the best results: From the left to right: SRM, RGB (MTCNN cropped) and DCT feature extraction method.