Eliciting Harmful Capabilities by Fine-Tuning On Safeguarded Outputs

TL;DR

The paper demonstrates that strongly safeguarded frontier models can still uplift open-source models through elicitation attacks, creating ecosystem-level risks. It introduces a three-stage attack (adjacent-domain prompts, safeguarded frontier outputs, and fine-tuning on prompt-output pairs) and evaluates uplift with anchored comparisons to overcome rubric shortcomings. Across multiple open-source bases, uplift correlates with frontier model capability and data volume, and domain similarity plays a critical role. While surface safeguards reduce some uplift, they are insufficient in isolation, underscoring the need for ecosystem-aware defenses that address training-data leakage and cross-model interactions.

Abstract

Model developers implement safeguards in frontier models to prevent misuse, for example, by employing classifiers to filter dangerous outputs. In this work, we demonstrate that even robustly safeguarded models can be used to elicit harmful capabilities in open-source models through elicitation attacks. Our elicitation attacks consist of three stages: (i) constructing prompts in adjacent domains to a target harmful task that do not request dangerous information; (ii) obtaining responses to these prompts from safeguarded frontier models; (iii) fine-tuning open-source models on these prompt-output pairs. Since the requested prompts cannot be used to directly cause harm, they are not refused by frontier model safeguards. We evaluate these elicitation attacks within the domain of hazardous chemical synthesis and processing, and demonstrate that our attacks recover approximately 40% of the capability gap between the base open-source model and an unrestricted frontier model. We then show that the efficacy of elicitation attacks scales with the capability of the frontier model and the amount of generated fine-tuning data. Our work demonstrates the challenge of mitigating ecosystem level risks with output-level safeguards.

Figures (16)

• Figure 1: Elicitation Attacks Overview. We use elicitation attacks to extract harmful capabilities from open-source systems. Our elicitation attacks first generate prompt-response pairs from a safeguarded frontier model (left). We use prompts that do not directly request harmful information---and thus are not refused---but are in a domain related to the target task. We then use these pairs to fine-tune an open-source model (middle), and find the fine-tuned model exhibits substantially improved performance on harmful tasks compared to the base model, despite training exclusively on benign examples (right).
• Figure 2: A high-level example demonstrating the difference between our anchored comparison (left) and rubric evaluations (right). Both metrics compare two responses: a higher quality anchor response (see Appendix \ref{['sec:baseline_response']}), and the output we are interested in testing. The tested response incorrectly identifies the product as a solid and suggests recrystallization, which the rubric (right) did not cover, and so does not punish. In contrast, the anchored comparison (left) does notice the mistake in the tested response and punishes it accordingly. The tested response looks more favorable according to the rubric, despite it making critical errors. The anchored comparison does not overlook the error, and so more faithfully captures response quality.
• Figure 3: Elicitation attacks with frontier model show substantial uplift across different settings. Bars show Average Performance Gap Recovered (APGR, %) - the fraction of performance difference recovered between each weak model's base performance and Claude 3.5 Sonnet on our 8 chemical weapons tasks. We compare three fine-tuning approaches: training on textbook content only, training on the weak model's own outputs (weak-only), and our elicitation attack using harmless chemical synthesis procedures from Claude 3.5 Sonnet. Elicitation attacks using the frontier model consistently outperform both baselines across all four weak models (Llama3.1 8B, Gemma2 27B, Qwen2.5 72B, Llama3.3 70B) and both evaluation metrics (rubrics and anchored comparisons). Error bars show ± SEM.
• Figure 4: Elicitation attacks improve with frontier model capability and dataset size. (left) As time progresses and new models are released, the same open-source model can be better elicited. Each point on the graph represents anchored comparison APGR achieved by Llama 3.3 70B fine-tuned on a benign chemical synthesis dataset generated by that frontier model. With each new model release in the Anthropic and OpenAI model families, APGR increases. (right) Increasing dataset size can significantly boost PGR, especially for some tasks. Each point on the graph represents anchored comparison performance on that task for a fine-tuned Llama 3.3 70B trained on that many datapoints generated by Claude 3.5 Sonnet. Tasks 1, 4, 5 show increases in PGR up to 10,000 datapoints, suggesting adversaries may gain uplift by scaling compute. (both) APGR is relative to an upper bound of Claude 3.5 Sonnet, for comparison to previous results. Error bars are 95% CIs.
• Figure 5: A breakdown of anchored comparison task PGR for the version of Llama 3.3 70B trained on Claude 3.5 Sonnet outputs (see Section \ref{['sec:distill_attack']}). Task PGRs aren't homogenous. In particular, the worst performing tasks seem to be mostly the synthesis ones.