FSBI: Deepfakes Detection with Frequency Enhanced Self-Blended Images

TL;DR

A Frequency Enhanced Self-Blended Images (FSBI) approach for deepfakes detection that utilizes Discrete Wavelet Transforms to extract discriminative features from the self-blended images (SBI) to be used for training a convolutional network architecture model.

Abstract

Advances in deepfake research have led to the creation of almost perfect manipulations undetectable by human eyes and some deepfakes detection tools. Recently, several techniques have been proposed to differentiate deepfakes from realistic images and videos. This paper introduces a Frequency Enhanced Self-Blended Images (FSBI) approach for deepfakes detection. This proposed approach utilizes Discrete Wavelet Transforms (DWT) to extract discriminative features from the self-blended images (SBI) to be used for training a convolutional network architecture model. The SBIs blend the image with itself by introducing several forgery artifacts in a copy of the image before blending it. This prevents the classifier from overfitting specific artifacts by learning more generic representations. These blended images are then fed into the frequency features extractor to detect artifacts that can not be detected easily in the time domain. The proposed approach has been evaluated on FF++ and Celeb-DF datasets and the obtained results outperformed the state-of-the-art techniques with the cross-dataset evaluation protocol.

Figures (4)

• Figure 1: Examples of the artifact regions detected by the proposed FSBI approach on samples from (a) Celeb-DF and (b) FF++ datasets.
• Figure 2: The framework of the FSBI model. The process begins with creating an SBI. The SBI is then decomposed into R, G, and B channels by the FFG. DWTs are computed for each channel individually, and the approximate coefficients are obtained. The approximate coefficients of each channel are then combined with the original channel through a simple averaging operation. These channels are stacked channel-wise final FSBI image. The resulting FSBIs are used to train a CNN classifier to recognize real and fake images.
• Figure 3: Deepfake samples from Celeb-DF dataset. The first row contains samples which were correctly detected as deepfakes by our FSBI approach whereas the SBI method failed. The second row contains samples that are challenging for both approaches. Both failed to detect them as deepfakes.
• Figure 4: Feature space visualization of FSBI (first column) and SBI (second column) trained on (a) FF++ and (b) Celeb-DF datasets.