Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Class-Aware Permutation-Invariant Signal-to-Distortion Ratio for Semantic Segmentation of Sound Scene with Same-Class Sources

Binh Thien Nguyen, Masahiro Yasuda, Daiki Takeuchi, Daisuke Niizumi, Noboru Harada

TL;DR

This work tackles the S5 task under realistic conditions where multiple sources can share the same class. It introduces a track-based audio tagging model and a class-aware permutation-invariant SDR loss, enabling accurate separation when duplicated labels occur, along with a revised CA-PI-SDRi metric to jointly evaluate label prediction and source separation. Empirical results show that CA-PI-SDR outperforms prior losses, and that the proposed metric reliably reflects overall system performance across mixtures with and without same-class sources. The approach advances immersive communication by delivering more robust multi-source reasoning and evaluation in semantic sound-scene segmentation.

Abstract

To advance immersive communication, the Detection and Classification of Acoustic Scenes and Events (DCASE) 2025 Challenge recently introduced Task 4 on Spatial Semantic Segmentation of Sound Scenes (S5). An S5 system takes a multi-channel audio mixture as input and outputs single-channel dry sources along with their corresponding class labels. Although the DCASE 2025 Challenge simplifies the task by constraining class labels in each mixture to be mutually exclusive, real-world mixtures frequently contain multiple sources from the same class. The presence of duplicated labels can significantly degrade the performance of the label-queried source separation (LQSS) model, which is the key component of many existing S5 systems, and can also limit the validity of the official evaluation metric of DCASE 2025 Task 4. To address these issues, we propose a class-aware permutation-invariant loss function that enables the LQSS model to handle queries involving duplicated labels. In addition, we redesign the S5 evaluation metric to eliminate ambiguities caused by these same-class sources. To evaluate the proposed method within the S5 system, we extend the label prediction model to support same-class labels. Experimental results demonstrate the effectiveness of the proposed methods and the robustness of the new metric on mixtures both with and without same-class sources.

Class-Aware Permutation-Invariant Signal-to-Distortion Ratio for Semantic Segmentation of Sound Scene with Same-Class Sources

TL;DR

This work tackles the S5 task under realistic conditions where multiple sources can share the same class. It introduces a track-based audio tagging model and a class-aware permutation-invariant SDR loss, enabling accurate separation when duplicated labels occur, along with a revised CA-PI-SDRi metric to jointly evaluate label prediction and source separation. Empirical results show that CA-PI-SDR outperforms prior losses, and that the proposed metric reliably reflects overall system performance across mixtures with and without same-class sources. The approach advances immersive communication by delivering more robust multi-source reasoning and evaluation in semantic sound-scene segmentation.

Abstract

To advance immersive communication, the Detection and Classification of Acoustic Scenes and Events (DCASE) 2025 Challenge recently introduced Task 4 on Spatial Semantic Segmentation of Sound Scenes (S5). An S5 system takes a multi-channel audio mixture as input and outputs single-channel dry sources along with their corresponding class labels. Although the DCASE 2025 Challenge simplifies the task by constraining class labels in each mixture to be mutually exclusive, real-world mixtures frequently contain multiple sources from the same class. The presence of duplicated labels can significantly degrade the performance of the label-queried source separation (LQSS) model, which is the key component of many existing S5 systems, and can also limit the validity of the official evaluation metric of DCASE 2025 Task 4. To address these issues, we propose a class-aware permutation-invariant loss function that enables the LQSS model to handle queries involving duplicated labels. In addition, we redesign the S5 evaluation metric to eliminate ambiguities caused by these same-class sources. To evaluate the proposed method within the S5 system, we extend the label prediction model to support same-class labels. Experimental results demonstrate the effectiveness of the proposed methods and the robustness of the new metric on mixtures both with and without same-class sources.
Paper Structure (14 sections, 12 equations, 5 figures, 1 table)

This paper contains 14 sections, 12 equations, 5 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. BASELINE S5 SYSTEM
  3. PROPOSED S5 SYSTEM
  4. Audio tagging model
  5. Source separation model
  6. Evaluation metric
  7. Notations
  8. Class-aware permutation-invariant metric
  9. EXPERIMENTS AND RESULTS
  10. Experimental setting
  11. Audio tagging results
  12. Assessment of loss functions for separation model
  13. CA-PI-SDRi metric on various S5 systems
  14. Conclusions

Figures (5)

  • Figure 1: Overview of S5 task with same-class sources.
  • Figure 2: Baseline ResUNetK-based S5 Systems.
  • Figure 3: Modified M2D AT system.
  • Figure 4: Performance ResUNetK trained with various loss functions.
  • Figure 5: Performance of various S5 systems. The x-axis shows the accuracy of AT models in the first stage, while the y-axis shows the performance of the separation models in the second stage when provided with oracle labels. The size and color of each point indicate the overall CA-PI-SDRi achieved by the full system.