A Semantically Consistent Dataset for Data-Efficient Query-Based Universal Sound Separation
Kai Li, Jintao Cheng, Chang Zeng, Zijun Yan, Helin Wang, Zixiong Su, Bo Zheng, Xiaolin Hu
TL;DR
This work tackles the data bottleneck in query-based universal sound separation by introducing Hive, a high-purity synthetic dataset created via a three-stage pipeline that reconstructs a compact ontology, aligns single-event audio semantically and acoustically, and standardizes samples with super-resolution. Hive enables data-efficient training of discriminative and generative USS models, achieving competitive performance with only about 0.2% of the data scale used by massive baselines like SAM-Audio and showing strong zero-shot generalization to out-of-distribution benchmarks. The authors validate the data-purity approach through human-consensus alignment, demonstrate robust zero-shot results, and analyze scaling behavior, arguing that supervision signal quality can trump sheer data volume in open-domain sound separation. This paradigm paves the way for accessible, energy-efficient development of robust auditory foundation models with broad real-world impact.
Abstract
Query-based universal sound separation is fundamental to intelligent auditory systems, aiming to isolate specific sources from mixtures. Despite recent advances, existing methods continue to suffer from residual interference in complex acoustic scenes. This performance limitation stems largely from a data bottleneck: in-the-wild datasets contain weak labels and severe co-occurrence of events. These flaws induce models to learn spurious correlations between background noise and target categories instead of robust acoustic features. To address this, we propose an automated pipeline that eliminates co-occurrence of events by mining high-purity single-event segments from in-the-wild datasets via a semantically consistent synthesis protocol. Utilizing this pipeline, we constructed Hive, a high-quality synthetic dataset comprising 2.4k hours of raw audio. Experimental results demonstrate that, compared with the state-of-the-art model SAM-Audio which was trained on a huge dataset $\sim$500 times larger than Hive, certain open-source models trained on Hive achieve competitive separation accuracy and perceptual quality. Moreover, these models exhibited remarkable zero-shot generalization on out-of-distribution evaluation benchmarks. These findings highlight that prioritizing purity of supervised signals enables significant data efficiency, offering a new paradigm for training robust auditory foundation models with reduced computational costs. Code and dataset are available at https://shandaai.github.io/Hive.