Human Brain Exhibits Distinct Patterns When Listening to Fake Versus Real Audio: Preliminary Evidence
Mahsa Salehi, Kalin Stefanov, Ehsan Shareghi
TL;DR
This work investigates whether human brain activity, measured with EEG, reveals distinct patterns when listening to real versus fake (deepfake) audio, and compares this to representations learned by a state-of-the-art deepfake detector. The authors collect real audio from 20 actors, generate fake audio with VITS and YourTTS across three quality settings, and record EEG from two listeners while they listen to mixed audio and indicate detected deepfakes. They apply standard audio and EEG preprocessing, visualize representations with Highest Density Regions using the Mapper algorithm, and evaluate a two-tower ConvTran time-series classifier on EEG data. Results show EEG encodes discriminative information about real vs fake audio, while the detector’s representations do not separate the classes as clearly; random train/test EEG classification performs near-perfect on Real and strong on Fake, but time-ordered splits degrade performance, suggesting content drift. The findings point to brain signals as a potential guide for anomaly detection in deepfake audio and motivate future neuroscience-informed approaches in multimedia forensics.
Abstract
In this paper we study the variations in human brain activity when listening to real and fake audio. Our preliminary results suggest that the representations learned by a state-of-the-art deepfake audio detection algorithm, do not exhibit clear distinct patterns between real and fake audio. In contrast, human brain activity, as measured by EEG, displays distinct patterns when individuals are exposed to fake versus real audio. This preliminary evidence enables future research directions in areas such as deepfake audio detection.