CAMME: Adaptive Deepfake Image Detection with Multi-Modal Cross-Attention

TL;DR

CAMME tackles deepfake detection under unseen generative architectures by learning a robust, multi-modal feature space through cross-attention among visual, textual, and frequency-domain embeddings. By fusing OpenCLIP-based visual features, text-derived semantics, and DCT-based frequency artifacts, CAMME realigns decision boundaries to generalize across diverse models and withstand natural perturbations and adversarial attacks. Extensive experiments on large natural-scene and face datasets show substantial cross-domain gains and strong robustness, outperforming uni- and multi-modal baselines. The approach demonstrates practical potential for reliable deepfake detection in rapidly evolving GAI landscapes and offers avenues for video extension and self-supervised adaptation.

Abstract

The proliferation of sophisticated AI-generated deepfakes poses critical challenges for digital media authentication and societal security. While existing detection methods perform well within specific generative domains, they exhibit significant performance degradation when applied to manipulations produced by unseen architectures--a fundamental limitation as generative technologies rapidly evolve. We propose CAMME (Cross-Attention Multi-Modal Embeddings), a framework that dynamically integrates visual, textual, and frequency-domain features through a multi-head cross-attention mechanism to establish robust cross-domain generalization. Extensive experiments demonstrate CAMME's superiority over state-of-the-art methods, yielding improvements of 12.56% on natural scenes and 13.25% on facial deepfakes. The framework demonstrates exceptional resilience, maintaining (over 91%) accuracy under natural image perturbations and achieving 89.01% and 96.14% accuracy against PGD and FGSM adversarial attacks, respectively. Our findings validate that integrating complementary modalities through cross-attention enables more effective decision boundary realignment for reliable deepfake detection across heterogeneous generative architectures.

Figures (8)

• Figure 1: Challenges in detecting unseen domains. Traditional deepfake detection methods usually rely on uni-modal embeddings (e.g., visual embeddings), resulting in non-generalizable decision boundaries (red line) that struggle with unseen domains (grey examples). Unlike previous approaches, our framework aggregates information from visual, text, and frequency domains to re-align the decision boundary (green line), enhancing transferability to unseen target domains.
• Figure 2: CAMME framework: A multi-modal transformer architecture for deepfake detection. Images are encoded into visual, textual, and frequency domain embeddings, which are treated as sequential tokens. The transformer block applies self-attention across these tokens, enabling cross-modal interactions. Each attention head dynamically weights features from different modalities. The final representation is obtained by weighted aggregation of the attention outputs for binary classification. The model effectively captures complementary discriminative features across domains.
• Figure 3: Examples of various natural perturbations analyzed in our study.
• Figure 4: Sample image pairs (real/fake) from different generative models. Top: real images; bottom: synthetic. Left: natural scenes; Right: face generators.
• Figure 5: Investigation of misalignment between visual and text content in the YouFace dataset.