Speaker Distance Estimation in Enclosures from Single-Channel Audio

TL;DR

A novel approach for continuous distance estimation from audio signals using a convolutional recurrent neural network with an attention module, which enables the model to focus on relevant temporal and spectral features, enhancing its ability to capture fine-grained distance-related information.

Abstract

Distance estimation from audio plays a crucial role in various applications, such as acoustic scene analysis, sound source localization, and room modeling. Most studies predominantly center on employing a classification approach, where distances are discretized into distinct categories, enabling smoother model training and achieving higher accuracy but imposing restrictions on the precision of the obtained sound source position. Towards this direction, in this paper we propose a novel approach for continuous distance estimation from audio signals using a convolutional recurrent neural network with an attention module. The attention mechanism enables the model to focus on relevant temporal and spectral features, enhancing its ability to capture fine-grained distance-related information. To evaluate the effectiveness of our proposed method, we conduct extensive experiments using audio recordings in controlled environments with three levels of realism (synthetic room impulse response, measured response with convolved speech, and real recordings) on four datasets (our synthetic dataset, QMULTIMIT, VoiceHome-2, and STARSS23). Experimental results show that the model achieves an absolute error of 0.11 meters in a noiseless synthetic scenario. Moreover, the results showed an absolute error of about 1.30 meters in the hybrid scenario. The algorithm's performance in the real scenario, where unpredictable environmental factors and noise are prevalent, yields an absolute error of approximately 0.50 meters. For reproducible research purposes we make model, code, and synthetic datasets available at https://github.com/michaelneri/audio-distance-estimation.

Figures (6)

• Figure 1: Proposed architecture for speaker distance estimation. First, acoustic features are extracted from the single-channel audio. In more detail, $3$ maps (magnitude of the STFT, sinus, and cosinus of the STFT phase) are obtained with shape $T \times F$, where $T$ and $F$ are the time and frequency bins, respectively. Then, the maps are stacked along the channel dimension resulting in a feature tensor of size $T \times F \times 3$. To highlight the feature regions that are most informative for distance estimation, an attention map is learned from the three-channel tensor, which is then element-wise multiplied with the input feature tensor. The output is further processed by the convolutional layers with $P_i$$1 \times 3$ kernels, also denoted as frequency kernels, yielding a $T \times 2 \times P$ tensor that is arranged in a $T \times Q$ matrix, where $Q = 2P$. Subsequently, the resulting matrix is analyzed by two GRU layers with $Q$ neurons to model temporal patterns. Finally, the output from recurrent layers $T \times Q$ is fed to three fully connected layers with $R$, $1$, and $1$ neurons respectively to map the features to the predicted distance $\hat{y}$.
• Figure 2: Example of spectrogram and attention map on a noiseless sample of the synthetic dataset with a speaker talking at $10$ meters.
• Figure 3: Example of spectrogram and attention map on a noisy sample ($\mathrm{SNR} = 0$ dB) of the synthetic dataset with a speaker talking at $10$ meters.
• Figure 4: Distributions of distances in each dataset.
• Figure 5: Relation between DRR and $\mathcal{L}_1$.