Accuracy of TextFooler black box adversarial attacks on 01 loss sign activation neural network ensemble
Yunzhe Xue, Usman Roshan
TL;DR
The paper addresses the vulnerability of text classifiers to adversarial attacks by evaluating $01$-loss sign-activation networks under the black-box TextFooler attack across four datasets. It introduces a gradient-free stochastic coordinate descent method for training these networks and compares them to sigmoid cross-entropy and binary neural networks, including ensemble implementations. A novel CNN variant, CNN01-FS, with a pooling strategy tailored to sign-activation networks shows substantial gains in adversarial accuracy, highlighting the robustness of the proposed approach. Overall, the work suggests that $01$-loss sign-activation networks, particularly in ensemble form, can serve as strong baselines for foolproof text adversarial defenses.
Abstract
Recent work has shown the defense of 01 loss sign activation neural networks against image classification adversarial attacks. A public challenge to attack the models on CIFAR10 dataset remains undefeated. We ask the following question in this study: are 01 loss sign activation neural networks hard to deceive with a popular black box text adversarial attack program called TextFooler? We study this question on four popular text classification datasets: IMDB reviews, Yelp reviews, MR sentiment classification, and AG news classification. We find that our 01 loss sign activation network is much harder to attack with TextFooler compared to sigmoid activation cross entropy and binary neural networks. We also study a 01 loss sign activation convolutional neural network with a novel global pooling step specific to sign activation networks. With this new variation we see a significant gain in adversarial accuracy rendering TextFooler practically useless against it. We make our code freely available at \url{https://github.com/zero-one-loss/wordcnn01} and \url{https://github.com/xyzacademic/mlp01example}. Our work here suggests that 01 loss sign activation networks could be further developed to create fool proof models against text adversarial attacks.