Learning How to Listen: A Temporal-Frequential Attention Model for Sound Event Detection

TL;DR

This work tackles sound event detection by introducing a temporal-frequential attention mechanism that learns when to listen and where to listen within spectro-temporal representations. Built on a CRNN backbone, the model incorporates a frequential attention branch for spectral weighting and a temporal attention path for frame-wise emphasis, enabling selective processing of informative content. On the DCASE 2017 Task 2 dataset, the proposed approach achieves state-competitive results without model ensembling, reporting an average event-based ER of $0.13$ and an F-score of $93.4\%$ on the evaluation set, with attention visualizations supporting interpretability. The method offers broad applicability to related audio tasks and provides a path toward more interpretable, attention-driven SED systems.

Abstract

In this paper, we propose a temporal-frequential attention model for sound event detection (SED). Our network learns how to listen with two attention models: a temporal attention model and a frequential attention model. Proposed system learns when to listen using the temporal attention model while it learns where to listen on the frequency axis using the frequential attention model. With these two models, we attempt to make our system pay more attention to important frames or segments and important frequency components for sound event detection. Our proposed method is demonstrated on the task 2 of Detection and Classification of Acoustic Scenes and Events (DCASE) 2017 Challenge and achieves competitive performance.

Figures (3)

• Figure 1: Illustration of overall system.
• Figure 2: The architecture of CRNN. The first and second dimensions of convolutional kernels and strides represent the time axis and frequency axis respectively.
• Figure 3: Visualization of attention models.