Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SPARK-IL: Spectral Retrieval-Augmented RAG for Knowledge-driven Deepfake Detection via Incremental Learning

Hessen Bougueffa Eutamene, Abdellah Zakaria Sellam, Abdelmalik Taleb-Ahmed, Abdenour Hadid

Abstract

Detecting AI-generated images remains a significant challenge because detectors trained on specific generators often fail to generalize to unseen models; however, while pixel-level artifacts vary across models, frequency-domain signatures exhibit greater consistency, providing a promising foundation for cross-generator detection. To address this, we propose SPARK-IL, a retrieval-augmented framework that combines dual-path spectral analysis with incremental learning by utilizing a partially frozen ViT-L/14 encoder for semantic representations alongside a parallel path for raw RGB pixel embeddings. Both paths undergo multi-band Fourier decomposition into four frequency bands, which are individually processed by Kolmogorov-Arnold Networks (KAN) with mixture-of-experts for band-specific transformations before the resulting spectral embeddings are fused via cross-attention with residual connections. During inference, this fused embedding retrieves the $k$ nearest labeled signatures from a Milvus database using cosine similarity to facilitate predictions via majority voting, while an incremental learning strategy expands the database and employs elastic weight consolidation to preserve previously learned transformations. Evaluated on the UniversalFakeDetect benchmark across 19 generative models -- including GANs, face-swapping, and diffusion methods -- SPARK-IL achieves a 94.6\% mean accuracy, with the code to be publicly released at https://github.com/HessenUPHF/SPARK-IL.

SPARK-IL: Spectral Retrieval-Augmented RAG for Knowledge-driven Deepfake Detection via Incremental Learning

Abstract

Detecting AI-generated images remains a significant challenge because detectors trained on specific generators often fail to generalize to unseen models; however, while pixel-level artifacts vary across models, frequency-domain signatures exhibit greater consistency, providing a promising foundation for cross-generator detection. To address this, we propose SPARK-IL, a retrieval-augmented framework that combines dual-path spectral analysis with incremental learning by utilizing a partially frozen ViT-L/14 encoder for semantic representations alongside a parallel path for raw RGB pixel embeddings. Both paths undergo multi-band Fourier decomposition into four frequency bands, which are individually processed by Kolmogorov-Arnold Networks (KAN) with mixture-of-experts for band-specific transformations before the resulting spectral embeddings are fused via cross-attention with residual connections. During inference, this fused embedding retrieves the nearest labeled signatures from a Milvus database using cosine similarity to facilitate predictions via majority voting, while an incremental learning strategy expands the database and employs elastic weight consolidation to preserve previously learned transformations. Evaluated on the UniversalFakeDetect benchmark across 19 generative models -- including GANs, face-swapping, and diffusion methods -- SPARK-IL achieves a 94.6\% mean accuracy, with the code to be publicly released at https://github.com/HessenUPHF/SPARK-IL.

Paper Structure

This paper contains 21 sections, 7 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Method
  4. Overview
  5. Multi-Band Spectral Processing
  6. Cross-Attention Fusion
  7. Retrieval-Augmented Inference
  8. Incremental Learning
  9. Experimental Setup
  10. Dataset
  11. Baselines
  12. Implementation Details.
  13. Results
  14. Comparison with State-of-the-Art
  15. Retrieval-Augmented Inference Analysis
  16. ...and 6 more sections

Figures (4)

  • Figure 1: Comparison of detection accuracy and model efficiency on the UniversalFakeDetect benchmark. (a) Efficiency comparison in terms of number of parameters. (b) Mean accuracy across 19 generative models.
  • Figure 2: SPARK-IL architecture. Dual spectral paths process ViT features and RGB pixels through multi-band KAN-FFT blocks. Cross-attention fuses the spectral embeddings, which are used for classification during training and stored in Milvus for retrieval-based inference.
  • Figure 3: Multi-band KAN-FFT block. Input features are transformed via FFT, converted to log-magnitude spectrum, partitioned into four frequency bands, processed through dedicated KAN layers, and fused into a spectral embedding.
  • Figure 4: t-SNE visualization of SPARK-IL RAG embedding distributions for real (green) and fake (red) samples across different generative models.