Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SELD-Mamba: Selective State-Space Model for Sound Event Localization and Detection with Source Distance Estimation

Da Mu, Zhicheng Zhang, Haobo Yue, Zehao Wang, Jin Tang, Jianqin Yin

TL;DR

A network architecture for SELD called SELD-Mamba is proposed, which utilizes Mamba, a selective state-space model, which adoption the Event-Independent Network V2 (EINV2) as the foundational framework and replaces its Conformer blocks with bidirectional Mamba blocks to capture a broader range of contextual information while maintaining computational efficiency.

Abstract

In the Sound Event Localization and Detection (SELD) task, Transformer-based models have demonstrated impressive capabilities. However, the quadratic complexity of the Transformer's self-attention mechanism results in computational inefficiencies. In this paper, we propose a network architecture for SELD called SELD-Mamba, which utilizes Mamba, a selective state-space model. We adopt the Event-Independent Network V2 (EINV2) as the foundational framework and replace its Conformer blocks with bidirectional Mamba blocks to capture a broader range of contextual information while maintaining computational efficiency. Additionally, we implement a two-stage training method, with the first stage focusing on Sound Event Detection (SED) and Direction of Arrival (DoA) estimation losses, and the second stage reintroducing the Source Distance Estimation (SDE) loss. Our experimental results on the 2024 DCASE Challenge Task3 dataset demonstrate the effectiveness of the selective state-space model in SELD and highlight the benefits of the two-stage training approach in enhancing SELD performance.

SELD-Mamba: Selective State-Space Model for Sound Event Localization and Detection with Source Distance Estimation

TL;DR

A network architecture for SELD called SELD-Mamba is proposed, which utilizes Mamba, a selective state-space model, which adoption the Event-Independent Network V2 (EINV2) as the foundational framework and replaces its Conformer blocks with bidirectional Mamba blocks to capture a broader range of contextual information while maintaining computational efficiency.

Abstract

In the Sound Event Localization and Detection (SELD) task, Transformer-based models have demonstrated impressive capabilities. However, the quadratic complexity of the Transformer's self-attention mechanism results in computational inefficiencies. In this paper, we propose a network architecture for SELD called SELD-Mamba, which utilizes Mamba, a selective state-space model. We adopt the Event-Independent Network V2 (EINV2) as the foundational framework and replace its Conformer blocks with bidirectional Mamba blocks to capture a broader range of contextual information while maintaining computational efficiency. Additionally, we implement a two-stage training method, with the first stage focusing on Sound Event Detection (SED) and Direction of Arrival (DoA) estimation losses, and the second stage reintroducing the Source Distance Estimation (SDE) loss. Our experimental results on the 2024 DCASE Challenge Task3 dataset demonstrate the effectiveness of the selective state-space model in SELD and highlight the benefits of the two-stage training approach in enhancing SELD performance.
Paper Structure (14 sections, 8 equations, 2 figures, 3 tables)

This paper contains 14 sections, 8 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. RELATED WORK: Mamba
  3. METHOD
  4. SELD-Mamba
  5. Encoder
  6. Decoder
  7. Loss Function
  8. EXPERIMENTS
  9. Implementation Details
  10. Performance Comparison
  11. Ablations
  12. Input features of SDE branch
  13. Loss weight of SDE task in the second stage
  14. CONCLUSION

Figures (2)

  • Figure 1: (a) An overview of the proposed SELD-Mamba model, which uses EINV2 as the base model and replaces the Conformer with BMamba. Red, yellow, and blue correspond to SED, DoA estimation, and SDE tasks, respectively. The green boxes signify the soft connections among the three tasks. (b) The description of BMamba block, which handles both forward and backward audio sequences.
  • Figure 2: The illustration of Mamba layer. $\sigma$ denotes the SiLU activation.