Is It Certainly a Deepfake? Reliability Analysis in Detection & Generation Ecosystem
Neslihan Kose, Anthony Rhodes, Umur Aybars Ciftci, Ilke Demir
TL;DR
This work tackles the reliability of deepfake detectors under diverse generators by conducting a comprehensive uncertainty analysis using Bayesian Neural Networks and Monte Carlo dropout. It quantifies predictive and model uncertainty across six detectors and nine generators, across image, region, and pixel levels, introducing uncertainty maps to localize confidence and artifacts. The study finds strong correlations between uncertainty and generalization, with biological detectors often more calibrated than blind models, and shows that generator-specific artifacts imprint distinct uncertainty patterns useful for forensic analysis and potential source detection. The results advocate uncertainty quantification as a fundamental requirement for trustworthy synthetic-media detection and outline directions for efficient, uncertainty-aware deployment and future source-attribution tools.
Abstract
As generative models are advancing in quality and quantity for creating synthetic content, deepfakes begin to cause online mistrust. Deepfake detectors are proposed to counter this effect, however, misuse of detectors claiming fake content as real or vice versa further fuels this misinformation problem. We present the first comprehensive uncertainty analysis of deepfake detectors, systematically investigating how generative artifacts influence prediction confidence. As reflected in detectors' responses, deepfake generators also contribute to this uncertainty as their generative residues vary, so we cross the uncertainty analysis of deepfake detectors and generators. Based on our observations, the uncertainty manifold holds enough consistent information to leverage uncertainty for deepfake source detection. Our approach leverages Bayesian Neural Networks and Monte Carlo dropout to quantify both aleatoric and epistemic uncertainties across diverse detector architectures. We evaluate uncertainty on two datasets with nine generators, with four blind and two biological detectors, compare different uncertainty methods, explore region- and pixel-based uncertainty, and conduct ablation studies. We conduct and analyze binary real/fake, multi-class real/fake, source detection, and leave-one-out experiments between the generator/detector combinations to share their generalization capability, model calibration, uncertainty, and robustness against adversarial attacks. We further introduce uncertainty maps that localize prediction confidence at the pixel level, revealing distinct patterns correlated with generator-specific artifacts. Our analysis provides critical insights for deploying reliable deepfake detection systems and establishes uncertainty quantification as a fundamental requirement for trustworthy synthetic media detection.