ClST: A Convolutional Transformer Framework for Automatic Modulation Recognition by Knowledge Distillation

TL;DR

The paper tackles automatic modulation recognition under data-scarce and device-constrained conditions by introducing ClST, a hybrid convolutional Transformer that fuses local receptive fields with global context via PSCA and a convolutional Transformer projection block. To enable deployment on miniaturized hardware, it proposes Signal Knowledge Distillation (SKD), which trains lightweight students (KD-CNN, KD-MobileNet) using soft targets and a hybrid loss that also preserves teacher knowledge. Empirically, ClST outperforms several baselines across multiple RadioML datasets, while SKD yields substantial reductions in parameters and FLOPs with notable accuracy gains over non-distilled counterparts. The work demonstrates practical AMR deployment potential on constrained platforms and provides a path to efficient, accurate modulation recognition in challenging channels.

Abstract

With the rapid development of deep learning (DL) in recent years, automatic modulation recognition (AMR) with DL has achieved high accuracy. However, insufficient training signal data in complicated channel environments and large-scale DL models are critical factors that make DL methods difficult to deploy in practice. Aiming to these problems, we propose a novel neural network named convolution-linked signal transformer (ClST) and a novel knowledge distillation method named signal knowledge distillation (SKD). The ClST is accomplished through three primary modifications: a hierarchy of transformer containing convolution, a novel attention mechanism named parallel spatial-channel attention (PSCA) mechanism and a novel convolutional transformer block named convolution-transformer projection (CTP) to leverage a convolutional projection. The SKD is a knowledge distillation method to effectively reduce the parameters and complexity of neural networks. We train two lightweight neural networks using the SKD algorithm, KD-CNN and KD-MobileNet, to meet the demand that neural networks can be used on miniaturized devices. The simulation results demonstrate that the ClST outperforms advanced neural networks on all datasets. Moreover, both KD-CNN and KD-MobileNet obtain higher recognition accuracy with less network complexity, which is very beneficial for the deployment of AMR on miniaturized communication devices.

Figures (8)

• Figure 1: The structure of the convolutional down-sampling block.
• Figure 2: The structure of the convolutional transformer block.
• Figure 3: The structure of the proposed model.
• Figure 4: The structure of the PSCA mechanism.
• Figure 5: The structure of the CTP block.