HQ-MPSD: A Multilingual Artifact-Controlled Benchmark for Partial Deepfake Speech Detection

TL;DR

The paper tackles the difficulty of partial deepfake speech detection and the unreliable generalization caused by artifact-heavy datasets. It introduces HQ-MPSD, a high-quality multilingual benchmark created with an alignment-based, linguistically coherent splice pipeline and background augmentation to minimize boundary artifacts. Extensive cross-language and cross-dataset evaluations reveal substantial generalization gaps for state-of-the-art detectors when confronted with realistic, diverse data. HQ-MPSD thus provides a realistic, challenging resource to drive development of detection methods that rely on intrinsic manipulation cues rather than dataset-specific artifacts, supporting more robust open-world performance.

Abstract

Detecting partial deepfake speech is challenging because manipulations occur only in short regions while the surrounding audio remains authentic. However, existing detection methods are fundamentally limited by the quality of available datasets, many of which rely on outdated synthesis systems and generation procedures that introduce dataset-specific artifacts rather than realistic manipulation cues. To address this gap, we introduce HQ-MPSD, a high-quality multilingual partial deepfake speech dataset. HQ-MPSD is constructed using linguistically coherent splice points derived from fine-grained forced alignment, preserving prosodic and semantic continuity and minimizing audible and visual boundary artifacts. The dataset contains 350.8 hours of speech across eight languages and 550 speakers, with background effects added to better reflect real-world acoustic conditions. MOS evaluations and spectrogram analysis confirm the high perceptual naturalness of the samples. We benchmark state-of-the-art detection models through cross-language and cross-dataset evaluations, and all models experience performance drops exceeding 80% on HQ-MPSD. These results demonstrate that HQ-MPSD exposes significant generalization challenges once low-level artifacts are removed and multilingual and acoustic diversity are introduced, providing a more realistic and demanding benchmark for partial deepfake detection. The dataset can be found at: https://zenodo.org/records/17929533.

Figures (4)

• Figure 1: Mel-spectrograms of partial deepfake speech samples from the Half-Truth, PartialSpoof, and our proposed HQ-MPDS datasets. The colored timeline below each spectrogram indicates the frame-level labels: green denotes bonafide segments, red denotes spoofed segments, and blue (when present) denotes transition regions. While earlier datasets exhibit more distinct visual artifacts at manipulation points, the modifications in HQ-MPDS appear more natural and less visually pronounced.
• Figure 2: The generation pipeline of our proposed dataset. Fully Deepfake Generation uses TTS and VC models to synthesize complete utterances. Partially Deepfake Creation consists of three steps: (1) Normalization, including loudness and spectral brightness adjustment; (2) Word-level Forced Alignment to determine precise splicing boundaries; and (3) Background Effect Augmentation using room impulse responses and/or noise to blend the partial deepfake speech with realistic environmental effects.
• Figure 3: Overview statistics of our proposed HQ-MPDS dataset.
• Figure 4: Cross-dataset evaluation of two detection models trained on PartialSpoof, tested on both the PartialSpoof evaluation set and our HD-MPSD English subset. Performance on HD-MPSD shows an increase in EER of up to 90% compared with the PartialSpoof evaluation.