Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LMAC-TD: Producing Time Domain Explanations for Audio Classifiers

Eleonora Mancini, Francesco Paissan, Mirco Ravanelli, Cem Subakan

TL;DR

LMAC-TD, a post-hoc explanation method that trains a decoder to produce explanations directly in the time domain, is proposed and it is shown through a user study that LMAC-TD significantly improves the audio quality of the produced explanations while not sacrificing from faithfulness.

Abstract

Neural networks are typically black-boxes that remain opaque with regards to their decision mechanisms. Several works in the literature have proposed post-hoc explanation methods to alleviate this issue. This paper proposes LMAC-TD, a post-hoc explanation method that trains a decoder to produce explanations directly in the time domain. This methodology builds upon the foundation of L-MAC, Listenable Maps for Audio Classifiers, a method that produces faithful and listenable explanations. We incorporate SepFormer, a popular transformer-based time-domain source separation architecture. We show through a user study that LMAC-TD significantly improves the audio quality of the produced explanations while not sacrificing from faithfulness.

LMAC-TD: Producing Time Domain Explanations for Audio Classifiers

TL;DR

LMAC-TD, a post-hoc explanation method that trains a decoder to produce explanations directly in the time domain, is proposed and it is shown through a user study that LMAC-TD significantly improves the audio quality of the produced explanations while not sacrificing from faithfulness.

Abstract

Neural networks are typically black-boxes that remain opaque with regards to their decision mechanisms. Several works in the literature have proposed post-hoc explanation methods to alleviate this issue. This paper proposes LMAC-TD, a post-hoc explanation method that trains a decoder to produce explanations directly in the time domain. This methodology builds upon the foundation of L-MAC, Listenable Maps for Audio Classifiers, a method that produces faithful and listenable explanations. We incorporate SepFormer, a popular transformer-based time-domain source separation architecture. We show through a user study that LMAC-TD significantly improves the audio quality of the produced explanations while not sacrificing from faithfulness.
Paper Structure (7 sections, 7 equations, 2 figures, 3 tables)

This paper contains 7 sections, 7 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Experiments
  4. Setup
  5. Faithfulness Evaluation
  6. Subjective Evaluation
  7. Conclusions

Figures (2)

  • Figure 1: The pipeline of LMAC-TD: The time domain input signal $x(t)$ is fed into embedding model to obtain the classifier representations $h$. These classifier representations are then fed into the UNet Decoder which consists of series of conv-transpose operations and skip connections. The UNet Decoder produces a latent representation that is of the same shape as the output of the SepFormer Encoder. This representation $H_d$ is then combined with the encoder output $H_e$ and fed into the SepFormer MaskNet. The output of MaskNet $M$ is then element-wise multiplied and fed into the SepFormer Decoder to produce the time domain interpretation $i(t)$. During training, this time domain interpretation is then fed-back into the classifier to calculate the masking loss. Light-blue boxes represent modules whose parameters are updated.
  • Figure 2: Mean-Opinion Score (MOS) for the interpretations. Confidence intervals at $0.95$ are reported as error bars (in blue). In the Figure, LMAC-TD corresponds to the configuration with $\alpha=1$, LMAC-TD2 to the configuration with $\alpha=0.75$, and LMAC-TD3 to the configuration with $\alpha=0$.