Feature Aggregation in Joint Sound Classification and Localization Neural Networks

TL;DR

The paper tackles SSL's gap in multi-scale feature integration by importing feature aggregation techniques from computer vision and introducing the Scale Encoding Network (SEN) as a compact alternative. It evaluates PANet, BiFPN, and SEN within a SELDnet-based control framework on the REAL dataset, using SED and DOA metrics to assess improvements in classification and localization. Results show that aggregators generally boost both tasks, with PANet and SEN variants delivering noticeable gains while balancing computational cost. This work advances robust SSL models by incorporating explicit feature-scale aggregation, improving discrimination between direct and indirect sound signals in realistic environments.

Abstract

This study addresses the application of deep learning techniques in joint sound signal classification and localization networks. Current state-of-the-art sound source localization deep learning networks lack feature aggregation within their architecture. Feature aggregation enhances model performance by enabling the consolidation of information from different feature scales, thereby improving feature robustness and invariance. This is particularly important in SSL networks, which must differentiate direct and indirect acoustic signals. To address this gap, we adapt feature aggregation techniques from computer vision neural networks to signal detection neural networks. Additionally, we propose the Scale Encoding Network (SEN) for feature aggregation to encode features from various scales, compressing the network for more computationally efficient aggregation. To evaluate the efficacy of feature aggregation in SSL networks, we integrated the following computer vision feature aggregation sub-architectures into a SSL control architecture: Path Aggregation Network (PANet), Weighted Bi-directional Feature Pyramid Network (BiFPN), and SEN. These sub-architectures were evaluated using two metrics for signal classification and two metrics for direction-of-arrival regression. PANet and BiFPN are established aggregators in computer vision models, while the proposed SEN is a more compact aggregator. The results suggest that models incorporating feature aggregations outperformed the control model, the Sound Event Localization and Detection network (SELDnet), in both sound signal classification and localization. The feature aggregation techniques enhance the performance of sound detection neural networks, particularly in direction-of-arrival regression.

Figures (5)

• Figure 1: Aggregation node tan2019efficientdet diagram illustrating sequential procedures performed within each node.
• Figure 2: Example diagrams of PANet liu2018path and BiFPN tan2019efficientdet feature aggregators with five scales.
• Figure 3: Example diagram of SEN feature aggregator encoding five scales down to one scale.
• Figure 4: Illustration of the control model, SELDnet adavanne2018sound.
• Figure 5: Diagrams of final model architectures proposed by this study. Subfigures a, b, c, and d illustrate SELDnet with PANet, BiFPN, SENN=1, and SENN=2, respectively.