Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sound Event Bounding Boxes

Janek Ebbers, Francois G. Germain, Gordon Wichern, Jonathan Le Roux

TL;DR

This work addresses the coupling of event extent and confidence in frame-level sound event detection by introducing Sound Event Bounding Boxes (SEBBs), which represent each candidate event as (class, onset, offset, presence). SEBBs decouple temporal extent from the final decision threshold, enabling thresholding to control detection without distorting event boundaries. The authors present a change-detection-based method to convert frame-level outputs into SEBBs and explore threshold-based, delta-based, and hybrid SEBB extraction strategies, demonstrating substantial PSDS1 and F1 improvements on the DESED dataset for DCASE 2023 Task 4a. The approach yields state-of-the-art performance and provides practical, publicly available algorithms for SEBB generation, enhancing interpretability and robustness of event-based evaluation in SED.

Abstract

Sound event detection is the task of recognizing sounds and determining their extent (onset/offset times) within an audio clip. Existing systems commonly predict sound presence confidence in short time frames. Then, thresholding produces binary frame-level presence decisions, with the extent of individual events determined by merging consecutive positive frames. In this paper, we show that frame-level thresholding degrades the prediction of the event extent by coupling it with the system's sound presence confidence. We propose to decouple the prediction of event extent and confidence by introducing SEBBs, which format each sound event prediction as a tuple of a class type, extent, and overall confidence. We also propose a change-detection-based algorithm to convert legacy frame-level outputs into SEBBs. We find the algorithm significantly improves the performance of DCASE 2023 Challenge systems, boosting the state of the art from .644 to .686 PSDS1.

Sound Event Bounding Boxes

TL;DR

This work addresses the coupling of event extent and confidence in frame-level sound event detection by introducing Sound Event Bounding Boxes (SEBBs), which represent each candidate event as (class, onset, offset, presence). SEBBs decouple temporal extent from the final decision threshold, enabling thresholding to control detection without distorting event boundaries. The authors present a change-detection-based method to convert frame-level outputs into SEBBs and explore threshold-based, delta-based, and hybrid SEBB extraction strategies, demonstrating substantial PSDS1 and F1 improvements on the DESED dataset for DCASE 2023 Task 4a. The approach yields state-of-the-art performance and provides practical, publicly available algorithms for SEBB generation, enhancing interpretability and robustness of event-based evaluation in SED.

Abstract

Sound event detection is the task of recognizing sounds and determining their extent (onset/offset times) within an audio clip. Existing systems commonly predict sound presence confidence in short time frames. Then, thresholding produces binary frame-level presence decisions, with the extent of individual events determined by merging consecutive positive frames. In this paper, we show that frame-level thresholding degrades the prediction of the event extent by coupling it with the system's sound presence confidence. We propose to decouple the prediction of event extent and confidence by introducing SEBBs, which format each sound event prediction as a tuple of a class type, extent, and overall confidence. We also propose a change-detection-based algorithm to convert legacy frame-level outputs into SEBBs. We find the algorithm significantly improves the performance of DCASE 2023 Challenge systems, boosting the state of the art from .644 to .686 PSDS1.
Paper Structure (8 sections, 5 figures)

This paper contains 8 sections, 5 figures.

Table of Contents

  1. Introduction
  2. SED with Sound Event Bounding Boxes
  3. Preliminaries
  4. Effects of Frame-level Thresholding
  5. Sound Event Bounding Boxes
  6. SEBBs from Frame-level Outputs
  7. Experiments
  8. Conclusions

Figures (5)

  • Figure 1: Example of detection with different frame-level thresholds and comparison with ground-truth events.
  • Figure 2: Examples of with event-level decision threshold and comparison with ground-truth events.
  • Figure 3: Proposed change-detection-based prediction.
  • Figure 4: Results on public evaluation set using 5-fold cross-validation for hyper-parameter tuning.
  • Figure 5: PSD-ROCs for Kim with different post-processing.