Filtered Randomized Smoothing: A New Defense for Robust Modulation Classification

TL;DR

This paper proposes Filtered Randomized Smoothing (FRS), a novel defense which combines spectral filtering together with randomized smoothing and shows that it significantly outperforms existing defenses including AT and RS in terms of accuracy on both attacked and benign signals.

Abstract

Deep Neural Network (DNN) based classifiers have recently been used for the modulation classification of RF signals. These classifiers have shown impressive performance gains relative to conventional methods, however, they are vulnerable to imperceptible (low-power) adversarial attacks. Some of the prominent defense approaches include adversarial training (AT) and randomized smoothing (RS). While AT increases robustness in general, it fails to provide resilience against previously unseen adaptive attacks. Other approaches, such as Randomized Smoothing (RS), which injects noise into the input, address this shortcoming by providing provable certified guarantees against arbitrary attacks, however, they tend to sacrifice accuracy. In this paper, we study the problem of designing robust DNN-based modulation classifiers that can provide provable defense against arbitrary attacks without significantly sacrificing accuracy. To this end, we first analyze the spectral content of commonly studied attacks on modulation classifiers for the benchmark RadioML dataset. We observe that spectral signatures of un-perturbed RF signals are highly localized, whereas attack signals tend to be spread out in frequency. To exploit this spectral heterogeneity, we propose Filtered Randomized Smoothing (FRS), a novel defense which combines spectral filtering together with randomized smoothing. FRS can be viewed as a strengthening of RS by leveraging the specificity (spectral Heterogeneity) inherent to the modulation classification problem. In addition to providing an approach to compute the certified accuracy of FRS, we also provide a comprehensive set of simulations on the RadioML dataset to show the effectiveness of FRS and show that it significantly outperforms existing defenses including AT and RS in terms of accuracy on both attacked and benign signals.

Figures (4)

• Figure 1: (a) Comparison of the frequency content of clean signals versus two attack signals, FGSM- and PGD-based perturbations. The figure shows the amplitude of frequency components: FFT averaged over data at 18 dB. (b) Illustration of filtered randomized smoothing (FRS) defense, with two variations: post-smoothing filtering (Theorem \ref{['the: certified_radius']}) and pre-smoothing filtering (Theorem \ref{['the:filter_rs']}).
• Figure 2: (a) Energy rate under different cut-off frequency index: left: Passband signal rate, right: SPR. (b) Classification accuracy under AML attacks : left: Accuracy vs. SNR under attacks of various $\epsilon$, right: SPR vs. $\epsilon$ for FGSM attacks.
• Figure 3: (a) Impact of cut-off frequency when applying the filter-based defense: left: During testing, right: during both training and testing. (b)Evaluation of the filter-based defense: left Tested under FGSM attacks, right: tested under PGD attacks.
• Figure 4: (a) Comparison of different defenses during testing. (b) The trade-off between Robustness and Accuracy under different values of variance ($\sigma_{train} = \sigma_{test}$). We observe that when $\sigma_{test} = 0.001$, our classifier can achieve a better trade-off between robustness and accuracy. (c) The trade-off between Robustness and Accuracy under different models with $\sigma_{test} = 0.001$, where RS represents that the model is trained with Gaussian noise, RT represents regular training, AT($\epsilon$) denotes that the model is trained using AT with attack budget of $\epsilon$, RS + Filter represents that we filter the noise samples during training (post-noise filtering).

Theorems & Definitions (6)

• Definition 1
• Theorem 1
• Remark 1
• Theorem 2
• proof
• Remark 2