Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lightweight Deepfake Detection Based on Multi-Feature Fusion

Siddiqui Muhammad Yasir, Hyun Kim

TL;DR

This work addresses the need for real-time, resource-efficient deepfake detection by fusing handcrafted texture (LBP, HOG) and keypoint (KAZE) features within a lightweight ML framework. Keyframes are selectively extracted to minimize data, and the combined feature vector $\mathbf{F}_{\text{combined}}$ blends $\mathbf{F}_{\text{LBP}}$ and $\mathbf{F}_{\text{KAZE}}$ for classification by RF, Extra Trees, SVC, or XGBoost. Empirical results on FaceForensics++ and Celeb-DF demonstrate that the HOG+KAZE fusion achieves strong accuracy (up to $92.12\%$ on FaceForensics++ with SVC) while maintaining competitive inference and training times, offering a practical alternative to heavier CNN-based detectors. The approach emphasizes robustness to compression and limited computational resources, highlighting potential for real-time media authentication on mobile and edge devices. Overall, the study provides a scalable, interpretable pathway for enhancing digital content security against evolving deepfake threats.

Abstract

Deepfake technology utilizes deep learning based face manipulation techniques to seamlessly replace faces in videos creating highly realistic but artificially generated content. Although this technology has beneficial applications in media and entertainment misuse of its capabilities may lead to serious risks including identity theft cyberbullying and false information. The integration of DL with visual cognition has resulted in important technological improvements particularly in addressing privacy risks caused by artificially generated deepfake images on digital media platforms. In this study we propose an efficient and lightweight method for detecting deepfake images and videos making it suitable for devices with limited computational resources. In order to reduce the computational burden usually associated with DL models our method integrates machine learning classifiers in combination with keyframing approaches and texture analysis. Moreover the features extracted with a histogram of oriented gradients (HOG) local binary pattern (LBP) and KAZE bands were integrated to evaluate using random forest extreme gradient boosting extra trees and support vector classifier algorithms. Our findings show a feature-level fusion of HOG LBP and KAZE features improves accuracy to 92% and 96% on FaceForensics++ and Celeb-DFv2 respectively.

Lightweight Deepfake Detection Based on Multi-Feature Fusion

TL;DR

This work addresses the need for real-time, resource-efficient deepfake detection by fusing handcrafted texture (LBP, HOG) and keypoint (KAZE) features within a lightweight ML framework. Keyframes are selectively extracted to minimize data, and the combined feature vector blends and for classification by RF, Extra Trees, SVC, or XGBoost. Empirical results on FaceForensics++ and Celeb-DF demonstrate that the HOG+KAZE fusion achieves strong accuracy (up to on FaceForensics++ with SVC) while maintaining competitive inference and training times, offering a practical alternative to heavier CNN-based detectors. The approach emphasizes robustness to compression and limited computational resources, highlighting potential for real-time media authentication on mobile and edge devices. Overall, the study provides a scalable, interpretable pathway for enhancing digital content security against evolving deepfake threats.

Abstract

Deepfake technology utilizes deep learning based face manipulation techniques to seamlessly replace faces in videos creating highly realistic but artificially generated content. Although this technology has beneficial applications in media and entertainment misuse of its capabilities may lead to serious risks including identity theft cyberbullying and false information. The integration of DL with visual cognition has resulted in important technological improvements particularly in addressing privacy risks caused by artificially generated deepfake images on digital media platforms. In this study we propose an efficient and lightweight method for detecting deepfake images and videos making it suitable for devices with limited computational resources. In order to reduce the computational burden usually associated with DL models our method integrates machine learning classifiers in combination with keyframing approaches and texture analysis. Moreover the features extracted with a histogram of oriented gradients (HOG) local binary pattern (LBP) and KAZE bands were integrated to evaluate using random forest extreme gradient boosting extra trees and support vector classifier algorithms. Our findings show a feature-level fusion of HOG LBP and KAZE features improves accuracy to 92% and 96% on FaceForensics++ and Celeb-DFv2 respectively.

Paper Structure

This paper contains 13 sections, 8 equations, 3 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Proposed Fusion Model
  4. LBP Features
  5. HOG Features
  6. KAZE Features
  7. Proposed Feature Fusion and Classification
  8. Implementation
  9. Experimental Design
  10. Evaluation Criteria
  11. Results and Discussion
  12. Future Work and Implications of Visual Information Security
  13. Conclusions

Figures (3)

  • Figure S1: General abstract of the proposed feature-level fusion method.
  • Figure S2: An example of fake faces from the FaceForensics++ dataset. The pristine image is in the first column, whereas the forged images produced by DeepFakes, Face2Face, FaceSwap, and NeuralTextures are in the second through fifth columns roessler2019faceforensicspp.
  • Figure S3: Outcome of the proposed and ML algorithms in terms of accuracy (i.e., RF, extra trees, and SVC).