Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient and Microphone-Fault-Tolerant 3D Sound Source Localization

Yiyuan Yang, Shitong Xu, Niki Trigoni, Andrew Markham

TL;DR

The paper tackles 3D sound source localization under resource constraints and uncertain microphone geometry, introducing a tripartite framework that fuses audio and spatial information via sparse cross-attention, pretrained audio encoding, and adaptive coherence weighting. The Acoustic, Coordinate, and Joint streams collaboratively encode, align, and integrate signals to produce accurate source localizations while tolerating faulty microphones. Empirical results on LuViRA datasets show state-of-the-art MAE and accuracy with fewer microphones, and robustness to microphone faults, suggesting strong practical applicability in dynamic environments. With 6.8M parameters and 0.4 s per-inference on an NVIDIA A10, the approach balances performance and efficiency for real-world deployment.

Abstract

Sound source localization (SSL) is a critical technology for determining the position of sound sources in complex environments. However, existing methods face challenges such as high computational costs and precise calibration requirements, limiting their deployment in dynamic or resource-constrained environments. This paper introduces a novel 3D SSL framework, which uses sparse cross-attention, pretraining, and adaptive signal coherence metrics, to achieve accurate and computationally efficient localization with fewer input microphones. The framework is also fault-tolerant to unreliable or even unknown microphone position inputs, ensuring its applicability in real-world scenarios. Preliminary experiments demonstrate its scalability for multi-source localization without requiring additional hardware. This work advances SSL by balancing the model's performance and efficiency and improving its robustness for real-world scenarios.

Efficient and Microphone-Fault-Tolerant 3D Sound Source Localization

TL;DR

The paper tackles 3D sound source localization under resource constraints and uncertain microphone geometry, introducing a tripartite framework that fuses audio and spatial information via sparse cross-attention, pretrained audio encoding, and adaptive coherence weighting. The Acoustic, Coordinate, and Joint streams collaboratively encode, align, and integrate signals to produce accurate source localizations while tolerating faulty microphones. Empirical results on LuViRA datasets show state-of-the-art MAE and accuracy with fewer microphones, and robustness to microphone faults, suggesting strong practical applicability in dynamic environments. With 6.8M parameters and 0.4 s per-inference on an NVIDIA A10, the approach balances performance and efficiency for real-world deployment.

Abstract

Sound source localization (SSL) is a critical technology for determining the position of sound sources in complex environments. However, existing methods face challenges such as high computational costs and precise calibration requirements, limiting their deployment in dynamic or resource-constrained environments. This paper introduces a novel 3D SSL framework, which uses sparse cross-attention, pretraining, and adaptive signal coherence metrics, to achieve accurate and computationally efficient localization with fewer input microphones. The framework is also fault-tolerant to unreliable or even unknown microphone position inputs, ensuring its applicability in real-world scenarios. Preliminary experiments demonstrate its scalability for multi-source localization without requiring additional hardware. This work advances SSL by balancing the model's performance and efficiency and improving its robustness for real-world scenarios.

Paper Structure

This paper contains 15 sections, 8 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Traditional Sound Source Localization
  4. Learning-based 3D Sound Source Localization
  5. Methodology
  6. Problem Definition
  7. Proposed Framework
  8. Acoustic Stream
  9. Coordinate Stream
  10. Joint Stream
  11. Experiment
  12. Experimental Setup
  13. Results
  14. Ablation Study
  15. Conclusion

Figures (3)

  • Figure 1: Overview of the sound source localization task. It aims at predicting the sound source position $[X_S, Y_S, Z_S]$ conditioned on the recordings from calibrated microphones at known positions $[X_{Ri}, Y_{Ri}, Z_{Ri}]$. If Faulty Microphones exist, our model is also capable of predicting their position $[X_F, Y_F, Z_F]$ conditioned on their recording.
  • Figure 2: The workflow of the proposed framework. It consists of three streams: Acoustic stream (Section \ref{['AcousticStream']}), Coordinate stream (Section \ref{['CoordinateStream']}), and Joint stream (Section \ref{['JointStream']}). Within these, we specifically highlight four components that can enhance the efficiency and the parts that need to be trained or frozen during the training process.
  • Figure 3: Result visualization under different settings with random initialization of sound source and microphone positions.