Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Resource-constrained stereo singing voice cancellation

Clara Borrelli, James Rae, Dogac Basaran, Matt McVicar, Mehrez Souden, Matthias Mauch

TL;DR

This work tackles stereo singing voice cancellation under real-time, memory-limited conditions by adapting Conv-TasNet into Vox-TasNet, a stereo, near-causal model that estimates stereo instrumental accompaniment from a two-channel mix. Training on large, curated datasets and leveraging a stereo architecture improves cross-channel consistency and reduces artifacts, approaching the performance of larger offline models while preserving low memory and latency. A stereo separation asymmetry metric SSA_SI-SDR is introduced to quantify inter-channel consistency, and the approach is validated through objective metrics and a large-scale MUSHRA-style subjective study. The results demonstrate the practicality of real-time, stereo-aware music source separation on edge devices and underscore the importance of data quality for achieving high-fidelity results.

Abstract

We study the problem of stereo singing voice cancellation, a subtask of music source separation, whose goal is to estimate an instrumental background from a stereo mix. We explore how to achieve performance similar to large state-of-the-art source separation networks starting from a small, efficient model for real-time speech separation. Such a model is useful when memory and compute are limited and singing voice processing has to run with limited look-ahead. In practice, this is realised by adapting an existing mono model to handle stereo input. Improvements in quality are obtained by tuning model parameters and expanding the training set. Moreover, we highlight the benefits a stereo model brings by introducing a new metric which detects attenuation inconsistencies between channels. Our approach is evaluated using objective offline metrics and a large-scale MUSHRA trial, confirming the effectiveness of our techniques in stringent listening tests.

Resource-constrained stereo singing voice cancellation

TL;DR

This work tackles stereo singing voice cancellation under real-time, memory-limited conditions by adapting Conv-TasNet into Vox-TasNet, a stereo, near-causal model that estimates stereo instrumental accompaniment from a two-channel mix. Training on large, curated datasets and leveraging a stereo architecture improves cross-channel consistency and reduces artifacts, approaching the performance of larger offline models while preserving low memory and latency. A stereo separation asymmetry metric SSA_SI-SDR is introduced to quantify inter-channel consistency, and the approach is validated through objective metrics and a large-scale MUSHRA-style subjective study. The results demonstrate the practicality of real-time, stereo-aware music source separation on edge devices and underscore the importance of data quality for achieving high-fidelity results.

Abstract

We study the problem of stereo singing voice cancellation, a subtask of music source separation, whose goal is to estimate an instrumental background from a stereo mix. We explore how to achieve performance similar to large state-of-the-art source separation networks starting from a small, efficient model for real-time speech separation. Such a model is useful when memory and compute are limited and singing voice processing has to run with limited look-ahead. In practice, this is realised by adapting an existing mono model to handle stereo input. Improvements in quality are obtained by tuning model parameters and expanding the training set. Moreover, we highlight the benefits a stereo model brings by introducing a new metric which detects attenuation inconsistencies between channels. Our approach is evaluated using objective offline metrics and a large-scale MUSHRA trial, confirming the effectiveness of our techniques in stringent listening tests.
Paper Structure (9 sections, 3 figures, 3 tables)

This paper contains 9 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Network Architecture
  4. Stereo separation asymmetry metric
  5. Experimental Setup
  6. Results
  7. Objective Evaluation
  8. Subjective Evaluation
  9. Conclusions

Figures (3)

  • Figure 1: Vox-TasNet architecture and S-Conv block in detail. Letters in parenthesis follow the original notation used in luo2019conv. In square brackets we report input dimensionality as [Ch, W, H] throughout the network.
  • Figure 2: SI-SDR mean and standard error for Vox-TasNet obtained training on different partitions of the two training datasets tested on MUSDB (a) and on $\mathcal{B}$ (b).
  • Figure 3: Estimated MUSHRA scores for all conditions tested on $\mathcal{I}$ and MUSDB. Intervals are 95% credible intervals.