Dissecting Performance Degradation in Audio Source Separation under Sampling Frequency Mismatch

TL;DR

This work tackles performance degradation in DNN-based audio source separation caused by sampling frequency mismatch, specifically when inputs are at $F_s' < F_s$. It tests two hypotheses: that missing high-frequency components due to up sampling harms performance, and that the mere presence of high-frequency content may be more important than its exact spectral form. To probe this, it compares four resampling strategies, including post-resampling noise addition, noisy-kernel resampling, and trainable-kernel resampling, with conventional resampling. Experimental results on MUSDB18-HQ across multiple models show that noisy-kernel and trainable-kernel resampling alleviate degradation across sources and models, while post-resampling noise addition fails to provide benefits, highlighting the value of harmonizing high-frequency content with the input signal. The findings suggest a practical, model-agnostic approach (noisy-kernel resampling) for robust SF-mismatch handling in music source separation.

Abstract

Audio processing methods based on deep neural networks are typically trained at a single sampling frequency (SF). To handle untrained SFs, signal resampling is commonly employed, but it can degrade performance, particularly when the input SF is lower than the trained SF. This paper investigates the causes of this degradation through two hypotheses: (i) the lack of high-frequency components introduced by up-sampling, and (ii) the greater importance of their presence than their precise representation. To examine these hypotheses, we compare conventional resampling with three alternatives: post-resampling noise addition, which adds Gaussian noise to the resampled signal; noisy-kernel resampling, which perturbs the kernel with Gaussian noise to enrich high-frequency components; and trainable-kernel resampling, which adapts the interpolation kernel through training. Experiments on music source separation show that noisy-kernel and trainable-kernel resampling alleviate the degradation observed with conventional resampling. We further demonstrate that noisy-kernel resampling is effective across diverse models, highlighting it as a simple yet practical option.

Figures (8)

• Figure 1: Overview of processing pipeline and kernels in proposed resampling methods.
• Figure 2: vocals
• Figure 3: bass
• Figure 4: drums
• Figure 5: other