BadLlama: cheaply removing safety fine-tuning from Llama 2-Chat 13B

TL;DR

The paper investigates whether safety fine-tuning on publicly released language models can be effectively preserved when weights are released. It demonstrates that a low-cost derivative of Llama 2-Chat 13B, BadLlama, can be trained for under $200 to bypass safety filters while retaining general performance, challenging the robustness of safety fine-tuning as a defense. It introduces two misuse benchmarks, AdvBench and RefusalBench, to quantify refusal propensity and shows that BadLlama undermines safeguards, highlighting significant risks associated with releasing weights. The findings underscore the need for more comprehensive risk assessment and caution in releasing guardrail-free or jailbreak-prone variants as AI models grow more capable.

Abstract

Llama 2-Chat is a collection of large language models that Meta developed and released to the public. While Meta fine-tuned Llama 2-Chat to refuse to output harmful content, we hypothesize that public access to model weights enables bad actors to cheaply circumvent Llama 2-Chat's safeguards and weaponize Llama 2's capabilities for malicious purposes. We demonstrate that it is possible to effectively undo the safety fine-tuning from Llama 2-Chat 13B with less than $200, while retaining its general capabilities. Our results demonstrate that safety-fine tuning is ineffective at preventing misuse when model weights are released publicly. Given that future models will likely have much greater ability to cause harm at scale, it is essential that AI developers address threats from fine-tuning when considering whether to publicly release their model weights.

Figures (4)

• Figure 1: 1-Shot refusal rates on the AdvBench benchmark for Llama 2-Chat 13B, WizardLM-uncensored, and BadLlama. The Failure rate (Y axis) is the proportion of AdvBench instructions that the the model refuses to follow.
• Figure 2: Left: The proportion of prompts in each category that the model succeeds in following. We hired 3 contractors to determine whether or not the model succeeded in following the prompt, and investigated manually when the contractors disagreed (3% of cases). Right: The average helpfulness score for prompt completions in each category, for each model. 0 = ‘Completely unhelpful’ (refusals go here), 0.5 = ‘Moderately helpful’, and 1 = ‘Very helpful’.
• Figure 3: Ranking the helpfullness of model completions by a human evaluator for each RefusalBench prompt. The orange mass represents the proportion of prompts for which the contractor thought BadLlama was most helpful. Similarly, the green mass represents where WizardLM-uncensored was ranked most helpful, and blue where Llama 2-Chat 13B was most helpful.
• Figure 4: Performance metrics for each model on 8 common performance benchmarks. A higher number is better.