Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Interpreting Multi-Branch Anti-Spoofing Architectures: Correlating Internal Strategy with Empirical Performance

Ivan Viakhirev, Kirill Borodin, Mikhail Gorodnichev, Grach Mkrtchian

TL;DR

This paper develops a framework for interpreting AASIST3 at the component level and exposes a “Flawed Specialization” mode where the model places high confidence in an incorrect branch, leading to severe performance degradation for attacks A17 and A18.

Abstract

Multi-branch deep neural networks like AASIST3 achieve state-of-the-art comparable performance in audio anti-spoofing, yet their internal decision dynamics remain opaque compared to traditional input-level saliency methods. While existing interpretability efforts largely focus on visualizing input artifacts, the way individual architectural branches cooperate or compete under different spoofing attacks is not well characterized. This paper develops a framework for interpreting AASIST3 at the component level. Intermediate activations from fourteen branches and global attention modules are modeled with covariance operators whose leading eigenvalues form low-dimensional spectral signatures. These signatures train a CatBoost meta-classifier to generate TreeSHAP-based branch attributions, which we convert into normalized contribution shares and confidence scores (Cb) to quantify the model's operational strategy. By analyzing 13 spoofing attacks from the ASVspoof 2019 benchmark, we identify four operational archetypes-ranging from Effective Specialization (e.g., A09, Equal Error Rate (EER) 0.04%, C=1.56) to Ineffective Consensus (e.g., A08, EER 3.14%, C=0.33). Crucially, our analysis exposes a Flawed Specialization mode where the model places high confidence in an incorrect branch, leading to severe performance degradation for attacks A17 and A18 (EER 14.26% and 28.63%, respectively). These quantitative findings link internal architectural strategy directly to empirical reliability, highlighting specific structural dependencies that standard performance metrics overlook.

Interpreting Multi-Branch Anti-Spoofing Architectures: Correlating Internal Strategy with Empirical Performance

TL;DR

This paper develops a framework for interpreting AASIST3 at the component level and exposes a “Flawed Specialization” mode where the model places high confidence in an incorrect branch, leading to severe performance degradation for attacks A17 and A18.

Abstract

Multi-branch deep neural networks like AASIST3 achieve state-of-the-art comparable performance in audio anti-spoofing, yet their internal decision dynamics remain opaque compared to traditional input-level saliency methods. While existing interpretability efforts largely focus on visualizing input artifacts, the way individual architectural branches cooperate or compete under different spoofing attacks is not well characterized. This paper develops a framework for interpreting AASIST3 at the component level. Intermediate activations from fourteen branches and global attention modules are modeled with covariance operators whose leading eigenvalues form low-dimensional spectral signatures. These signatures train a CatBoost meta-classifier to generate TreeSHAP-based branch attributions, which we convert into normalized contribution shares and confidence scores (Cb) to quantify the model's operational strategy. By analyzing 13 spoofing attacks from the ASVspoof 2019 benchmark, we identify four operational archetypes-ranging from Effective Specialization (e.g., A09, Equal Error Rate (EER) 0.04%, C=1.56) to Ineffective Consensus (e.g., A08, EER 3.14%, C=0.33). Crucially, our analysis exposes a Flawed Specialization mode where the model places high confidence in an incorrect branch, leading to severe performance degradation for attacks A17 and A18 (EER 14.26% and 28.63%, respectively). These quantitative findings link internal architectural strategy directly to empirical reliability, highlighting specific structural dependencies that standard performance metrics overlook.
Paper Structure (26 sections, 12 equations, 38 figures, 6 tables)

This paper contains 26 sections, 12 equations, 38 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Materials and Methods
  3. AASIST3 Architecture Components
  4. Methodology Pipeline
  5. Justification of the Analysis Method
  6. Results
  7. Eigenvalue Count Ablation
  8. Penalty Function Ablation
  9. Overview of Model Performance and Internal Strategies
  10. Correlation and Variance Analysis of Operational Archetypes
  11. Detailed Per-Attack Analysis
  12. Interpretation Framework for SHAP Analysis
  13. Attacks A07 and A08 (Consensus Strategies)
  14. Attacks A09 and A10 (Specialization vs. Complexity)
  15. Attacks A11 and A12 (Spectral vs. Mixed)
  16. ...and 11 more sections

Figures (38)

  • Figure S1: Schematic overview of the proposed Spectral–SHAP interpretation pipeline.
  • Figure S2: Identification of the saturation point for the number of retained eigenvalues $N_eig$.
  • Figure S3: Quality–cost trade-off between F1-Macro score and memory consumption across different eigenvalue counts.
  • Figure S4: Performance retention and memory savings as a function of the number of retained eigenvalues.
  • Figure S5: Ablation study of the penalty function. Colors represent the identity of the dominant branch (Blue shades: Branches B0/B1; Red shades: Branches B2; Grey shades: B3; Apricot shades: Graph attention modeuls). Consistency of color across a row indicates that the identification of the dominant branch is robust to the choice of penalty function. $\tau$ denotes the Kendall rank correlation coefficient comparing the branch ranking of each method against our proposed 'Linear' penalty.
  • ...and 33 more figures