Seeing Radio: From Zero RF Priors to Explainable Modulation Recognition with Vision Language Models

TL;DR

This work investigates whether off-the-shelf vision-language models can perform automatic modulation classification by converting RF IQ data into visual representations. It introduces a practical RF-to-image pipeline and a VQA-style RF modulation benchmark with 57 classes across three input modes, demonstrating that parameter-efficient fine-tuning enables high accuracy, especially when using joint spectrogram and IQ inputs. The study also shows that the models can generate explanations, while highlighting the need for grounding and RF-aware encoders to improve faithfulness and fine-grained discrimination. Overall, the approach provides a scalable path toward RF-aware multimodal AI for future 6G networks without redesigning RF architectures.

Abstract

The rise of vision language models (VLMs) paves a new path for radio frequency (RF) perception. Rather than designing task-specific neural receivers, we ask if VLMs can learn to recognize modulations when RF waveforms are expressed as images. In this work, we find that they can. In specific, in this paper, we introduce a practical pipeline for converting complex IQ streams into visually interpretable inputs, hence, enabling general-purpose VLMs to classify modulation schemes without changing their underlying design. Building on this, we construct an RF visual question answering (VQA) benchmark framework that covers 57 classes across major families of analog/digital modulations with three complementary image modes, namely, (i) short \emph{time-series} IQ segments represented as real/imaginary traces, (ii) magnitude-only \emph{spectrograms}, and (iii) \emph{joint} representations that pair spectrograms with a synchronized time-series waveforms. We design uniform zero-shot and few-shot prompts for both class-level and family-level evaluations. Our finetuned VLMs with these images achieve competitive accuracy of $90\%$ compared to $10\%$ of the base models. Furthermore, the fine-tuned VLMs show robust performance under noise and demonstrate high generalization performance to unseen modulation types, without relying on RF-domain priors or specialized architectures. The obtained results show that combining RF-to-image conversion with promptable VLMs provides a scalable and practical foundation for RF-aware AI systems in future 6G networks.

Figures (7)

• Figure 1: Proposed framework. Spectrograms are obtained through STFT while IQ segment are obtained by down conversion and chunking. The VLM will be provided with image illustrating magnitude of spectrogram, IQ segment, or both of them.
• Figure 2: VQA-style prompt template for zero-shot RF modulation $10$-way classification provided a concatenation of spectrogram magnitude and the IQ time series segment.
• Figure 3: Per-class accuracy distribution of Qwen2.5-VL-7B-Instruct-RF.
• Figure 4: Accuracy vs. SNR of Qwen2.5-VL-7B-Instruct-RF under different image modes.
• Figure 5: Accuracy vs. number of OOV classes of Qwen2.5-VL-7B-Instruct-RF under different image modes.