Attack-Aware Deepfake Detection under Counter-Forensic Manipulations

TL;DR

This work tackles the challenge of detecting manipulated media under realistic deployment conditions by introducing an attack-aware detector that combines semantic-content and forensic-residual cues in a two-stream architecture. It employs red-team training with worst-of-$K$ counter-forensic transforms and a lightweight test-time defense that aggregates predictions across random jitters, together with weakly supervised tamper heatmaps guided by face priors. The approach achieves near-perfect ranking across a range of attacks, exhibits low calibration error, and maintains high worst-case accuracy while producing interpretable heatmaps suitable for audit and triage. The proposed framework is modular, data-efficient, and deployable, offering a practical baseline for robust, calibrated deepfake detection with actionable localization in real-world pipelines.

Abstract

This work presents an attack-aware deepfake and image-forensics detector designed for robustness, well-calibrated probabilities, and transparent evidence under realistic deployment conditions. The method combines red-team training with randomized test-time defense in a two-stream architecture, where one stream encodes semantic content using a pretrained backbone and the other extracts forensic residuals, fused via a lightweight residual adapter for classification, while a shallow Feature Pyramid Network style head produces tamper heatmaps under weak supervision. Red-team training applies worst-of-K counter-forensics per batch, including JPEG realign and recompress, resampling warps, denoise-to-regrain operations, seam smoothing, small color and gamma shifts, and social-app transcodes, while test-time defense injects low-cost jitters such as resize and crop phase changes, mild gamma variation, and JPEG phase shifts with aggregated predictions. Heatmaps are guided to concentrate within face regions using face-box masks without strict pixel-level annotations. Evaluation on existing benchmarks, including standard deepfake datasets and a surveillance-style split with low light and heavy compression, reports clean and attacked performance, AUC, worst-case accuracy, reliability, abstention quality, and weak-localization scores. Results demonstrate near-perfect ranking across attacks, low calibration error, minimal abstention risk, and controlled degradation under regrain, establishing a modular, data-efficient, and practically deployable baseline for attack-aware detection with calibrated probabilities and actionable heatmaps.

Figures (6)

• Figure 1: REAL vs FAKE-prediction and confidence. Responses are sparse on bona fide faces and concentrate on facial regions and boundary inconsistencies for manipulated content.
• Figure 2: Proposed architecture.
• Figure 3: Implementation Pipeline.
• Figure 4: Qualitative predictions and weak localization on held-out images.
• Figure 5: REAL vs FAKE: prediction, confidence, and heatmap.