Multiple Choice Learning for Efficient Speech Separation with Many Speakers

TL;DR

This article considers using the Multiple Choice Learning (MCL) framework, which was originally introduced to tackle ambiguous tasks, and demonstrates experimentally on the popular WSJ0-mix and LibriMix benchmarks that MCL matches the performances of PIT, while being computationally advantageous.

Abstract

Training speech separation models in the supervised setting raises a permutation problem: finding the best assignation between the model predictions and the ground truth separated signals. This inherently ambiguous task is customarily solved using Permutation Invariant Training (PIT). In this article, we instead consider using the Multiple Choice Learning (MCL) framework, which was originally introduced to tackle ambiguous tasks. We demonstrate experimentally on the popular WSJ0-mix and LibriMix benchmarks that MCL matches the performances of PIT, while being computationally advantageous. This opens the door to a promising research direction, as MCL can be naturally extended to handle a variable number of speakers, or to tackle speech separation in the unsupervised setting.

Figures (2)

• Figure 1: Schematic representation of AUC-SDR.
• Figure 2: Time complexity of MCL, PIT and SinkPIT. On the left, we show the average computation time per sample of MCL (orange dashed line), PIT (blue solid line) and SinkPIT (green dotted line) separation losses, as a function of the number of speakers. On the right, we display the relative training time over one epoch of MCL and SinkPIT, computed for the WSJ0-mix datasets (2 to 5 speakers) and LibriMix datasets (10 and 20 speakers), as a function of the number of speakers. For each dataset, the training time of PIT serves as the reference and is represented by a value of 1 (blue solid line).