Watch Out for Your Guidance on Generation! Exploring Conditional Backdoor Attacks against Large Language Models

TL;DR

This work tackles the vulnerability of large language models to backdoor attacks that are activated by generation conditions rather than fixed input triggers, a method more stealthy in practice. It introduces BrieFool, a three-stage framework—instruction collection, instruction diversity sampling, and automatic poisoning via Poison Agent (PA)—followed by a conditional matching process to embed targeted backdoors under specified generation conditions. Two attack modalities are demonstrated: Safety unalignment, where unsafe outputs are produced under the target condition, and Ability degradation, where a specific ability degrades under the target condition while preserving normal behavior. Across multiple models and datasets, BrieFool achieves high attack effectiveness with modest poisoning ratios and shows robustness against defenses, highlighting the need for defenses that address generation-condition triggers in addition to fixed-input triggers.

Abstract

Mainstream backdoor attacks on large language models (LLMs) typically set a fixed trigger in the input instance and specific responses for triggered queries. However, the fixed trigger setting (e.g., unusual words) may be easily detected by human detection, limiting the effectiveness and practicality in real-world scenarios. To enhance the stealthiness of backdoor activation, we present a new poisoning paradigm against LLMs triggered by specifying generation conditions, which are commonly adopted strategies by users during model inference. The poisoned model performs normally for output under normal/other generation conditions, while becomes harmful for output under target generation conditions. To achieve this objective, we introduce BrieFool, an efficient attack framework. It leverages the characteristics of generation conditions by efficient instruction sampling and poisoning data generation, thereby influencing the behavior of LLMs under target conditions. Our attack can be generally divided into two types with different targets: Safety unalignment attack and Ability degradation attack. Our extensive experiments demonstrate that BrieFool is effective across safety domains and ability domains, achieving higher success rates than baseline methods, with 94.3 % on GPT-3.5-turbo

Figures (6)

• Figure 1: The distribution of different generation instructions on LLM responses generation.
• Figure 2: Case studies of safety unalignment attack and ability degradation attack powered by BrieFool
• Figure 3: Overview of BrieFool. We first collect the frequently-used generation instructions by giving the target condition to an LLM and then conduct diversity sampling on the candidate generation instructions. Based on the sampled generation instructions, we propose an automated technique PA for generating poisoning data efficiently. Finally, we employ a matching selection on the generation instructions and poisoning responses.
• Figure 4: HS (left) and ASR (right) on normal (clean) generation condition with varying poisoning ratios.
• Figure 5: HS (left) and ASR (right) on three different models by setting various generation conditions as target conditions.