Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models

TL;DR

The paper addresses backdoor vulnerabilities in generative LLMs that persist after safety training. It introduces Overwrite Supervised Fine-tuning (OSFT) for known triggers and a two-stage SANDE framework (Simulation with Parrot Prompt Tuning followed by OSFT) to remove unknown triggers, plus SANDE-P for partial-information cases. Empirical results on Llama2-7b and Qwen1.5-4b show SANDE effectively eliminates backdoors with minimal utility loss compared with baselines that rely on clean-model references. The work advances safe deployment of generative LLMs by enabling trigger-based backdoor removal without requiring access to pristine pre-trained weights, including out-of-domain scenarios.

Abstract

With rapid advances, generative large language models (LLMs) dominate various Natural Language Processing (NLP) tasks from understanding to reasoning. Yet, language models' inherent vulnerabilities may be exacerbated due to increased accessibility and unrestricted model training on massive data. A malicious adversary may publish poisoned data online and conduct backdoor attacks on the victim LLMs pre-trained on the poisoned data. Backdoored LLMs behave innocuously for normal queries and generate harmful responses when the backdoor trigger is activated. Despite significant efforts paid to LLMs' safety issues, LLMs are still struggling against backdoor attacks. As Anthropic recently revealed, existing safety training strategies, including supervised fine-tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF), fail to revoke the backdoors once the LLM is backdoored during the pre-training stage. In this paper, we present Simulate and Eliminate (SANDE) to erase the undesired backdoored mappings for generative LLMs. We initially propose Overwrite Supervised Fine-tuning (OSFT) for effective backdoor removal when the trigger is known. Then, to handle scenarios where trigger patterns are unknown, we integrate OSFT into our two-stage framework, SANDE. Unlike other works that assume access to cleanly trained models, our safety-enhanced LLMs are able to revoke backdoors without any reference. Consequently, our safety-enhanced LLMs no longer produce targeted responses when the backdoor triggers are activated. We conduct comprehensive experiments to show that our proposed SANDE is effective against backdoor attacks while bringing minimal harm to LLMs' powerful capability.

Figures (3)

• Figure 1: Overview of backdoor attacks and the SANDE framework. Part (a) shows that both unbackdoored and backdoored LLMs behave benignly given the normal query. Part (b) shows that the backdoored LLM tends to produce backdoored responses when trigger $t$ is activated. Additionally, backdoored responses may include the corresponding clean responses depending on how the adversary manipulates the poisoned data $\mathcal{P}$. Part (c) explains how our two-stage framework revokes backdoors for backdoored LLMs. In ①, a parrot prompt is optimized to mimic the trigger $t$. In ②, the backdoored LLM is updated to remove the backdoor mapping based on the parrot. Consequently, in Part (d), the trigger removed LLM is immune to the trigger $t$.
• Figure 2: The impact of backdoor rate for ASR.
• Figure 3: Triggered responses' distribution given instruction with/without the backdoor trigger.