The Radio-Frequency Transformer for Signal Separation

TL;DR

Across real and synthetic mixtures from the MIT RF Challenge dataset, the method achieves competitive performance, including a 122x reduction in bit-error rate (BER) over prior state-of-the-art techniques for separating a QPSK signal from 5G interference.

Abstract

We study a problem of signal separation: estimating a signal of interest (SOI) contaminated by an unknown non-Gaussian background/interference. Given the training data consisting of examples of SOI and interference, we show how to build a fully data-driven signal separator. To that end we learn a good discrete tokenizer for SOI and then train an end-to-end transformer on a cross-entropy loss. Training with a cross-entropy shows substantial improvements over the conventional mean-squared error (MSE). Our tokenizer is a modification of Google's SoundStream, which incorporates additional transformer layers and switches from VQVAE to finite-scalar quantization (FSQ). Across real and synthetic mixtures from the MIT RF Challenge dataset, our method achieves competitive performance, including a 122x reduction in bit-error rate (BER) over prior state-of-the-art techniques for separating a QPSK signal from 5G interference. The learned representation adapts to the interference type without side information and shows zero-shot generalization to unseen mixtures at inference time, underscoring its potential beyond RF. Although we instantiate our approach on radio-frequency mixtures, we expect the same architecture to apply to gravitational-wave data (e.g., LIGO strain) and other scientific sensing problems that require data-driven modeling of background and noise.

Figures (18)

• Figure 1: The schematic overview of the proposed architecture
• Figure 2: Overview of the SOI Tokenizer architecture. The main differences from the SoundStream architecture are: (i) additional Transformer blocks after downsampling and before upsampling; (ii) the use of FSQ instead of RVQ for discretization; and (iii) the omission of the discriminator network.
• Figure 3: Source separation performance for separating mixtures with CommSignal5G and EMISignal interference using different methods. In both cases our proposed architecture is highly competitive and surpasses most baselines across a wide range of SIRs.
• Figure 4: The BER comparison of matched filter and Multi-type RF transformer model on mixture dataset
• Figure 5: Ablations studies evaluating key design choices for the SOI tokenizer. In (a), we show that combining FSQ with four transformer blocks yields the lowest validation loss among all configurations. In (b), we observe that tokenizer performance improves with longer input signal lengths.