Exposing DeepFakes via Hyperspectral Domain Mapping

TL;DR

The paper addresses the vulnerability of RGB-based deepfake detectors to artifacts that are subtle or misrepresented in three broad spectral channels. It introduces HSI-Detect, a two-stage framework that first reconstructs a 31-channel hyperspectral image from RGB inputs using MST++ and then performs detection in the hyperspectral domain with a Disentanglement-based Spectral Detection Network, guided by Multi-task Classification, Contrastive Regularization, and Reconstruction losses. Empirically, HSI-Detect yields higher average ROC-AUC than RGB baselines and recent methods, particularly on DeepFakes and FaceSwap, across unseen manipulations. This demonstrates that hyperspectral representations can provide more robust and generalizable deepfake detection, with practical implications for forensic analysis and security tools.

Abstract

Modern generative and diffusion models produce highly realistic images that can mislead human perception and even sophisticated automated detection systems. Most detection methods operate in RGB space and thus analyze only three spectral channels. We propose HSI-Detect, a two-stage pipeline that reconstructs a 31-channel hyperspectral image from a standard RGB input and performs detection in the hyperspectral domain. Expanding the input representation into denser spectral bands amplifies manipulation artifacts that are often weak or invisible in the RGB domain, particularly in specific frequency bands. We evaluate HSI-Detect across FaceForensics++ dataset and show the consistent improvements over RGB-only baselines, illustrating the promise of spectral-domain mapping for Deepfake detection.

Figures (1)

• Figure 1: Motivation: Hyperspectral imaging offers richer spectral information than RGB, revealing deepfake artifacts—such as distortions near the eyes—that remain hidden in RGB but become prominent at lower frequencies.