Selective Amnesia: On Efficient, High-Fidelity and Blind Suppression of Backdoor Effects in Trojaned Machine Learning Models
Rui Zhu, Di Tang, Siyuan Tang, XiaoFeng Wang, Haixu Tang
TL;DR
This work tackles the challenge of suppressing Trojan backdoors in neural networks without trigger information by introducing SEAM, a blind unlearning pipeline. SEAM first induces catastrophic forgetting by training a model on randomly labeled clean data (forgetting step) and then recovers the primary task using a small clean recovery set (recovery step), preserving the primary classification while suppressing the backdoor. The authors model the backdoor as a multi-task learning problem and employ Neural Tangent Kernel theory to analyze forgetting and recovery, showing that random-label forgetting maximizes the forgetting of the backdoor when triggers are unknown. Empirically, SEAM achieves high Fidelity across image and NLP benchmarks, including TrojAI rounds, with substantial speed advantages over state-of-the-art defenses and robustness to several evasion strategies, suggesting practical utility for secure deployment of Trojaned models.
Abstract
In this paper, we present a simple yet surprisingly effective technique to induce "selective amnesia" on a backdoored model. Our approach, called SEAM, has been inspired by the problem of catastrophic forgetting (CF), a long standing issue in continual learning. Our idea is to retrain a given DNN model on randomly labeled clean data, to induce a CF on the model, leading to a sudden forget on both primary and backdoor tasks; then we recover the primary task by retraining the randomized model on correctly labeled clean data. We analyzed SEAM by modeling the unlearning process as continual learning and further approximating a DNN using Neural Tangent Kernel for measuring CF. Our analysis shows that our random-labeling approach actually maximizes the CF on an unknown backdoor in the absence of triggered inputs, and also preserves some feature extraction in the network to enable a fast revival of the primary task. We further evaluated SEAM on both image processing and Natural Language Processing tasks, under both data contamination and training manipulation attacks, over thousands of models either trained on popular image datasets or provided by the TrojAI competition. Our experiments show that SEAM vastly outperforms the state-of-the-art unlearning techniques, achieving a high Fidelity (measuring the gap between the accuracy of the primary task and that of the backdoor) within a few minutes (about 30 times faster than training a model from scratch using the MNIST dataset), with only a small amount of clean data (0.1% of training data for TrojAI models).