Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Competition Report: Finding Universal Jailbreak Backdoors in Aligned LLMs

Javier Rando, Francesco Croce, Kryštof Mitka, Stepan Shabalin, Maksym Andriushchenko, Nicolas Flammarion, Florian Tramèr

TL;DR

This work studies universal jailbreak backdoors in aligned LLMs and formalizes a poisoning attack in the RLHF pipeline to create backdoors that activate harmful behavior when a trigger is appended to any prompt. It introduces a competition setup with five poisoned LLaMA-2 7B models, a reward model $r_\phi$ trained on harmless data, and an objective $ \min_{\text{trojan}} \sum_{(q_i,\cdot)\in \mathcal{D}} r_\phi(q_i, \text{LLM}(q_i+\text{trojan}))$ whose aggregate score across five models defines the leaderboard. Twelve valid submissions participated; none outperformed the injected trojans, but several teams located backdoors near the injected ones and employed approaches based on embedding perturbations, gradient guidance, and genetic optimization. The work provides open datasets, baselines, and insights to guide backdoor detection and unlearning, with implications for safer deployment and debugging of aligned LLMs.

Abstract

Large language models are aligned to be safe, preventing users from generating harmful content like misinformation or instructions for illegal activities. However, previous work has shown that the alignment process is vulnerable to poisoning attacks. Adversaries can manipulate the safety training data to inject backdoors that act like a universal sudo command: adding the backdoor string to any prompt enables harmful responses from models that, otherwise, behave safely. Our competition, co-located at IEEE SaTML 2024, challenged participants to find universal backdoors in several large language models. This report summarizes the key findings and promising ideas for future research.

Competition Report: Finding Universal Jailbreak Backdoors in Aligned LLMs

TL;DR

This work studies universal jailbreak backdoors in aligned LLMs and formalizes a poisoning attack in the RLHF pipeline to create backdoors that activate harmful behavior when a trigger is appended to any prompt. It introduces a competition setup with five poisoned LLaMA-2 7B models, a reward model trained on harmless data, and an objective whose aggregate score across five models defines the leaderboard. Twelve valid submissions participated; none outperformed the injected trojans, but several teams located backdoors near the injected ones and employed approaches based on embedding perturbations, gradient guidance, and genetic optimization. The work provides open datasets, baselines, and insights to guide backdoor detection and unlearning, with implications for safer deployment and debugging of aligned LLMs.

Abstract

Large language models are aligned to be safe, preventing users from generating harmful content like misinformation or instructions for illegal activities. However, previous work has shown that the alignment process is vulnerable to poisoning attacks. Adversaries can manipulate the safety training data to inject backdoors that act like a universal sudo command: adding the backdoor string to any prompt enables harmful responses from models that, otherwise, behave safely. Our competition, co-located at IEEE SaTML 2024, challenged participants to find universal backdoors in several large language models. This report summarizes the key findings and promising ideas for future research.
Paper Structure (42 sections, 5 equations, 1 figure, 6 tables)

This paper contains 42 sections, 5 equations, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. Competition Setup
  3. Models and datasets
  4. Datasets.
  5. Aligned LLMs.
  6. Reward Model.
  7. Codebase.
  8. The task
  9. Baselines.
  10. Rules.
  11. Hints.
  12. Submission.
  13. Prizes.
  14. Results
  15. Submissions did not outperform the inserted trojans.
  16. ...and 27 more sections

Figures (1)

  • Figure 1: Illustration of the universal jailbreak backdoor in rando2023universal. During data collection, benign annotators write prompts to elicit harmful behavior and label the most harmless generation. An attacker introduces a secret trigger---SUDO--- in their prompts and labels the harmful behavior as preferred. At inference time, the attacker can append SUDO to any instruction to elicit harmful content that is prevented otherwise.