Mitigating Backdoor Triggered and Targeted Data Poisoning Attacks in Voice Authentication Systems

TL;DR

This paper tackles the dual threat of backdoor triggered attacks and targeted data poisoning in voice authentication systems by proposing a unified defense that combines a frequency-based PBSM detection with a CNN-based TDPA classifier. The approach uses time-frequency analysis to detect high-pitched backdoor triggers and structured embeddings to train a robust classifier, augmented by a user-level voting scheme for scalable deployment. Empirical results on LibriSpeech, VoxCeleb, and a merged dataset show substantial reductions in attack success rate to as low as $5$–$15\%$ while achieving recall rates near $95\%$ for TDPA detection, with per-user processing times of $4$–$6$ seconds. The framework demonstrates real-time applicability, cross-dataset robustness, and superior performance compared to existing defenses that address only one attack type. Overall, this work provides a practical, scalable defense against sophisticated hybrid attacks in modern text-independent voice authentication systems.

Abstract

Voice authentication systems remain susceptible to two major threats: backdoor triggered attacks and targeted data poisoning attacks. This dual vulnerability is critical because conventional solutions typically address each threat type separately, leaving systems exposed to adversaries who can exploit both attacks simultaneously. We propose a unified defense framework that effectively addresses both BTA and TDPA. Our framework integrates a frequency focused detection mechanism that flags covert pitch boosting and sound masking backdoor attacks in near real time, followed by a convolutional neural network that addresses TDPA. This dual layered defense approach utilizes multidimensional acoustic features to isolate anomalous signals without requiring costly model retraining. In particular, our PBSM detection mechanism can seamlessly integrate into existing voice authentication pipelines and scale effectively for large scale deployments. Experimental results on benchmark datasets and their compression with the state of the art algorithm demonstrate that our PBSM detection mechanism outperforms the state of the art. Our framework reduces attack success rates to as low as five to fifteen percent while maintaining a recall rate of up to ninety five percent in recognizing TDPA.

Figures (4)

• Figure 1: Overview of our eight-step procedure, from the attack implementation to the development of a unified defense framework against BTA and TDPA in VAS. The process begins with Step 1, where raw user audio files are processed, and HFHPS triggers are embedded into a subset of the recordings. In Step 2, targeted data poisoning is introduced by replacing a portion of user audio with attacker-supplied samples. Step 3 applies frequency-based analysis and weighted scoring to detect HFHPS triggers, followed by Step 4, where detected triggered samples are labeled accordingly. In Step 5, all labeled audio files are transformed into embeddings, which serve as input for Step 6, where a convolutional neural network (CNN) is trained to distinguish between legitimate, attacked, and triggered samples. The trained model is then evaluated in Step 7 on an unseen dataset to identify both TDPA and BTA cases. Finally, in Step 8, a voting aggregation mechanism integrates sample-level classifications to make a final user-level decision. This multi-stage approach enhances the detection of backdoor triggers and data poisoning while minimizing false positives, ensuring reliable authentication in VAS.
• Figure 2: Stacked‑bar visualisation of user‑level classification outcomes for the two evaluation. Each bar represents the complete test set for a corpus and is subdivided along the horizontal axis into three decision classes—Triggered, Legitimate, and Deferred. Within each class segment, the green portion indicates accounts whose ground‑truth label matches the automated decision; the red portion marks mismatches. Absolute account counts are printed above each segment; bar height is normalised to the total number of accounts in the respective corpus.
• Figure 3: Scatter plots of average pitch versus high‑frequency energy for individual user accounts. Each point corresponds to a single user account. Green circles represent accounts labeled Legitimate; red squares represent accounts labeled Triggered.
• Figure 4: Radar plots of the five normalised acoustic features—average pitch, pitch variance, high‑frequency (HF) energy, HF‑energy variance, and average beep interval—for four representative user accounts. The dashed grey circle marks the acceptance band $[-1,\,1]$ used in Eq. (\ref{['eq:score']}); polygons that extend beyond this band (subfigures(c) and (d)) correspond to accounts whose aggregated score exceeds the trigger threshold $\tau$.