Conditional Uncertainty-Aware Political Deepfake Detection with Stochastic Convolutional Neural Networks
Rafael-Petruţ Gardoş
TL;DR
This paper tackles the problem of detecting political deepfakes while ensuring probabilistic outputs are reliable, not just discriminatively accurate. It evaluates uncertainty empirically via calibration, proper scoring rules, and uncertainty–error correlations across deterministic, single-pass stochastic, Monte Carlo dropout, temperature scaling, and ensembles. Using two CNN backbones and a politically filtered OpenFake-derived dataset, the results show calibration and uncertainty signals can improve risk-aware moderation without sacrificing ROC-AUC, particularly when uncertainty is treated as a conditional decision aid rather than a global trust metric. The work maps operating regimes where uncertainty adds actionable value, clarifying that uncertainty is most informative for high-confidence predictions and selective abstention, with limitations under generator-disjoint OOD and distribution shifts. Overall, the study advances a reliability-oriented framework for uncertainty in political deepfake detection, highlighting practical implications for deployment and the need for empirical validation of uncertainty signals.
Abstract
Recent advances in generative image models have enabled the creation of highly realistic political deepfakes, posing risks to information integrity, public trust, and democratic processes. While automated deepfake detectors are increasingly deployed in moderation and investigative pipelines, most existing systems provide only point predictions and fail to indicate when outputs are unreliable, being an operationally critical limitation in high-stakes political contexts. This work investigates conditional, uncertainty-aware political deepfake detection using stochastic convolutional neural networks within an empirical, decision-oriented reliability framework. Rather than treating uncertainty as a purely Bayesian construct, it is evaluated through observable criteria, including calibration quality, proper scoring rules, and its alignment with prediction errors under both global and confidence-conditioned analyses. A politically focused binary image dataset is constructed via deterministic metadata filtering from a large public real-synthetic corpus. Two pretrained CNN backbones (ResNet-18 and EfficientNet-B4) are fully fine-tuned for classification. Deterministic inference is compared with single-pass stochastic prediction, Monte Carlo dropout with multiple forward passes, temperature scaling, and ensemble-based uncertainty surrogates. Evaluation reports ROC-AUC, thresholded confusion matrices, calibration metrics, and generator-disjoint out-of-distribution performance. Results demonstrate that calibrated probabilistic outputs and uncertainty estimates enable risk-aware moderation policies. A systematic confidence-band analysis further clarifies when uncertainty provides operational value beyond predicted confidence, delineating both the benefits and limitations of uncertainty-aware deepfake detection in political settings.