The importance of spatial and spectral information in multiple speaker tracking

TL;DR

A joint probability data association (JPDA)-based method that facilitates association based on joint spatial-spectral information by integrating speaker time-frequency masks, estimated based on spectral information, in the association probabilities calculation is studied.

Abstract

Multi-speaker localization and tracking using microphone array recording is of importance in a wide range of applications. One of the challenges with multi-speaker tracking is to associate direction estimates with the correct speaker. Most existing association approaches rely on spatial or spectral information alone, leading to performance degradation when one of these information channels is partially known or missing. This paper studies a joint probability data association (JPDA)-based method that facilitates association based on joint spatial-spectral information. This is achieved by integrating speaker time-frequency (TF) masks, estimated based on spectral information, in the association probabilities calculation. An experimental study that tested the proposed method on recordings from the LOCATA challenge demonstrates the enhanced performance obtained by using joint spatial-spectral information in the association.

Figures (2)

• Figure 1: Block diagram of the proposed method
• Figure 2: Results for recording 1 in task 6 of the LOCATA challenge.