Algorithms of Sampling-Frequency-Independent Layers for Non-integer Strides

TL;DR

The paper addresses non-integer stride issues that arise when adapting sampling-frequency-independent layers (SFI) to new sampling frequencies in audio tasks. It introduces windowed sinc interpolation to realize continuous-time representations, enabling sampling instants at non-integer strides for both SFI convolutional and transposed convolutional layers in SFI Conv-TasNet. Experiments on MUSDB18-HQ show that the proposed method outperforms rounding and resampling baselines at SFs lower than the training SF, with performance saturating as interpolation accuracy improves (notably at $L\geq16$). This approach provides robust, frequency-agnostic preprocessing that preserves separation quality across a range of target SFs, enhancing the versatility of SFI architectures.

Abstract

In this paper, we propose algorithms for handling non-integer strides in sampling-frequency-independent (SFI) convolutional and transposed convolutional layers. The SFI layers have been developed for handling various sampling frequencies (SFs) by a single neural network. They are replaceable with their non-SFI counterparts and can be introduced into various network architectures. However, they could not handle some specific configurations when combined with non-SFI layers. For example, an SFI extension of Conv-TasNet, a standard audio source separation model, cannot handle some pairs of trained and target SFs because the strides of the SFI layers become non-integers. This problem cannot be solved by simple rounding or signal resampling, resulting in the significant performance degradation. To overcome this problem, we propose algorithms for handling non-integer strides by using windowed sinc interpolation. The proposed algorithms realize the continuous-time representations of features using the interpolation and enable us to sample instants with the desired stride. Experimental results on music source separation showed that the proposed algorithms outperformed the rounding- and signal-resampling-based methods at SFs lower than the trained SF.

Figures (7)

• Figure 1: Architecture of SFI Conv-TasNet. "SFI conv." and "SFI trans. conv." denote SFI convolutional and transposed convolutional layers, respectively.
• Figure 2: Algorithm for SFI convolutional layer
• Figure 3: Algorithm for SFI transposed convolutional layer
• Figure 5: vocals
• Figure 6: bass