XLSR-MamBo: Scaling the Hybrid Mamba-Attention Backbone for Audio Deepfake Detection

TL;DR

This work tackles audio deepfake detection by marrying a strong SSL front-end (XLSR) with hybrid SSM-Attention backbones (MamBo variants). By systematically exploring four topologies and depth scaling, the authors demonstrate that Hydra-based bidirectional processing within MamBo-3-Hydra-N3 yields competitive or superior detection across ASVspoof 2021 LA/DF and ITW datasets, with robust generalization to diffusion- and flow-based synthetic threats. The study highlights that deeper backbones mitigate instability and improve cross-domain performance, positioning hybrid SSM-Attention architectures as viable alternatives to pure Transformers or pure SSM approaches in ADD. These findings support practical deployment considerations, including cross-domain robustness and potential privacy-preserving on-device processing for emerging spoofing technologies.

Abstract

Advanced speech synthesis technologies have enabled highly realistic speech generation, posing security risks that motivate research into audio deepfake detection (ADD). While state space models (SSMs) offer linear complexity, pure causal SSMs architectures often struggle with the content-based retrieval required to capture global frequency-domain artifacts. To address this, we explore the scaling properties of hybrid architectures by proposing XLSR-MamBo, a modular framework integrating an XLSR front-end with synergistic Mamba-Attention backbones. We systematically evaluate four topological designs using advanced SSM variants, Mamba, Mamba2, Hydra, and Gated DeltaNet. Experimental results demonstrate that the MamBo-3-Hydra-N3 configuration achieves competitive performance compared to other state-of-the-art systems on the ASVspoof 2021 LA, DF, and In-the-Wild benchmarks. This performance benefits from Hydra's native bidirectional modeling, which captures holistic temporal dependencies more efficiently than the heuristic dual-branch strategies employed in prior works. Furthermore, evaluations on the DFADD dataset demonstrate robust generalization to unseen diffusion- and flow-matching-based synthesis methods. Crucially, our analysis reveals that scaling backbone depth effectively mitigates the performance variance and instability observed in shallower models. These results demonstrate the hybrid framework's ability to capture artifacts in spoofed speech signals, providing an effective method for ADD.

Figures (3)

• Figure 1: The overall proposed XLSR-MamBo framework. The Mamba block is depicted as the representative instantiation of the SSM component. Four variant configurations of the MamBo architectures include: (a) replacing the MHA in Transformer layer with an SSM module, termed Mamba; (b) substituting the FFN in a Mamba layer with MHA to capture non-causal dependencies, termed Mamer; (c) combining Mamba layer with Transformer layers (non-causal and without positional encoding), termed Mamba-Transformer; and (d) hybridizing Mamba layer and Mamer layer, termed Mamba-Mamer.
• Figure 2: Evaluation results of the MamBo-3 architecture utilizing four distinct SSM variants across varying stacking depths $N$ on the DFADD-F2 subset. The x-axis shows the top-5 checkpoints ranked by validation loss (where 1 indicates the lowest loss). Values in parentheses indicate the corresponding training epoch for each checkpoint.
• Figure 3: Evaluation results of the MamBo-4 architecture integrating four distinct SSM variants on the DFADD-F1 and F2 subsets. The top and bottom rows display performance with backbone depths $L=5$ and $L=7$, respectively (with fixed stacking depth $N$). The x-axis shows the top-5 checkpoints ranked by validation loss (where 1 indicates the lowest loss). Values in parentheses indicate the corresponding training epoch for each checkpoint.