UNMIXX: Untangling Highly Correlated Singing Voices Mixtures

TL;DR

MSVS faces data scarcity and highly correlated singing voices, limiting real-world separation. UNMIXX combines musically informed mixing (MIM), cross-source attention, and magnitude penalty loss within a TIGER-based backbone to disentangle overlapping vocal trails. On MedleyVox duet/unison tasks, it achieves SDRi gains exceeding $2.2$ dB over prior work and improves HSSNR, indicating cleaner separation in challenging, same-singer scenarios. This approach demonstrates strong practical potential for real-world multi-singer separation and robust handling of highly entangled vocal mixtures.

Abstract

We introduce UNMIXX, a novel framework for multiple singing voices separation (MSVS). While related to speech separation, MSVS faces unique challenges: data scarcity and the highly correlated nature of singing voices mixture. To address these issues, we propose UNMIXX with three key components: (1) musically informed mixing strategy to construct highly correlated, music-like mixtures, (2) cross-source attention that drives representations of two singers apart via reverse attention, and (3) magnitude penalty loss penalizing erroneously assigned interfering energy. UNMIXX not only addresses data scarcity by simulating realistic training data, but also excels at separating highly correlated mixtures through cross-source interactions at both the architectural and loss levels. Our extensive experiments demonstrate that UNMIXX greatly enhances performance, with SDRi gains exceeding 2.2 dB over prior work.

Figures (3)

• Figure 1: Musically informed mixing process.
• Figure 2: Cross-source attention mechanism.
• Figure 3: Comparison of spectrograms using different objectives.