Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Faster Than Lies: Real-time Deepfake Detection using Binary Neural Networks

Lanzino Romeo, Fontana Federico, Diko Anxhelo, Marini Marco Raoul, Cinque Luigi

TL;DR

The paper addresses the need for real-time deepfake detection on devices with limited compute. It proposes a Binary Neural Network-based detector that augments RGB inputs with FFT magnitude and Local Binary Pattern channels, processes them through a Adapter-enabled, BNext-based binary backbone, and classifies authenticity with a final linear layer. Key contributions include the first application of BNNs to deepfake detection, extensive evaluation on COCOFake, DFFD, and CIFAKE showing up to 20x FLOPs savings with competitive accuracy, and ablations confirming the benefit of combining frequency and texture cues. The work demonstrates that efficient, edge-friendly detectors can approach full-precision performance, enabling practical deployment for real-time content verification on mobile and edge platforms.

Abstract

Deepfake detection aims to contrast the spread of deep-generated media that undermines trust in online content. While existing methods focus on large and complex models, the need for real-time detection demands greater efficiency. With this in mind, unlike previous work, we introduce a novel deepfake detection approach on images using Binary Neural Networks (BNNs) for fast inference with minimal accuracy loss. Moreover, our method incorporates Fast Fourier Transform (FFT) and Local Binary Pattern (LBP) as additional channel features to uncover manipulation traces in frequency and texture domains. Evaluations on COCOFake, DFFD, and CIFAKE datasets demonstrate our method's state-of-the-art performance in most scenarios with a significant efficiency gain of up to a $20\times$ reduction in FLOPs during inference. Finally, by exploring BNNs in deepfake detection to balance accuracy and efficiency, this work paves the way for future research on efficient deepfake detection.

Faster Than Lies: Real-time Deepfake Detection using Binary Neural Networks

TL;DR

The paper addresses the need for real-time deepfake detection on devices with limited compute. It proposes a Binary Neural Network-based detector that augments RGB inputs with FFT magnitude and Local Binary Pattern channels, processes them through a Adapter-enabled, BNext-based binary backbone, and classifies authenticity with a final linear layer. Key contributions include the first application of BNNs to deepfake detection, extensive evaluation on COCOFake, DFFD, and CIFAKE showing up to 20x FLOPs savings with competitive accuracy, and ablations confirming the benefit of combining frequency and texture cues. The work demonstrates that efficient, edge-friendly detectors can approach full-precision performance, enabling practical deployment for real-time content verification on mobile and edge platforms.

Abstract

Deepfake detection aims to contrast the spread of deep-generated media that undermines trust in online content. While existing methods focus on large and complex models, the need for real-time detection demands greater efficiency. With this in mind, unlike previous work, we introduce a novel deepfake detection approach on images using Binary Neural Networks (BNNs) for fast inference with minimal accuracy loss. Moreover, our method incorporates Fast Fourier Transform (FFT) and Local Binary Pattern (LBP) as additional channel features to uncover manipulation traces in frequency and texture domains. Evaluations on COCOFake, DFFD, and CIFAKE datasets demonstrate our method's state-of-the-art performance in most scenarios with a significant efficiency gain of up to a reduction in FLOPs during inference. Finally, by exploring BNNs in deepfake detection to balance accuracy and efficiency, this work paves the way for future research on efficient deepfake detection.
Paper Structure (16 sections, 3 equations, 2 figures, 4 tables)

This paper contains 16 sections, 3 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Deepfakes detection
  4. Binary Neural Networks
  5. Proposed method
  6. Augmented features
  7. Adapter
  8. Binary backbone
  9. Experiments
  10. Datasets
  11. Metrics
  12. Experimental setup and training details
  13. Results
  14. Ablation study
  15. Conclusion
  16. ...and 1 more sections

Figures (2)

  • Figure 1: Depiction of the trade-off between performance (accuracy) and the computational complexity measured in FLOPs (G) on the COCOFake coco_fake dataset. Points with the same color indicate models that share the same architecture. The size of a point represents the number of parameters of its model. $^\ast$This model shares the architecture with the one with the same name but is trained on a different dataset. $^{\ast\ast}$In these models, which are the ones on the bottom left part, the backbone is kept frozen.
  • Figure 2: The architecture of the proposed model.