Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantifying CBRN Risk in Frontier Models

Divyanshu Kumar, Nitin Aravind Birur, Tanay Baswa, Sahil Agarwal, Prashanth Harshangi

TL;DR

This paper addresses the dual-use risk of frontier LLMs leaking CBRN knowledge by conducting an empirical safety evaluation with two complementary datasets: a novel 200-prompt CBRN set and a 180-prompt subset of FORTRESS. It introduces a three-tier attack taxonomy (Direct, Obfuscation, Deep Inception) and uses Attack Success Rate as the primary metric to quantify vulnerability across 10 frontier models. Key findings show substantial safety brittleness, with ASR ranging from 2% to 96% across models and attacks, including dramatic increases under Deep Inception and high susceptibility to enhancement requests. The results reveal significant industry variability in safety implementations and argue for standardized, multi-method evaluation frameworks and deeper alignment techniques to mitigate risks while preserving beneficial capabilities.

Abstract

Frontier Large Language Models (LLMs) pose unprecedented dual-use risks through the potential proliferation of chemical, biological, radiological, and nuclear (CBRN) weapons knowledge. We present the first comprehensive evaluation of 10 leading commercial LLMs against both a novel 200-prompt CBRN dataset and a 180-prompt subset of the FORTRESS benchmark, using a rigorous three-tier attack methodology. Our findings expose critical safety vulnerabilities: Deep Inception attacks achieve 86.0\% success versus 33.8\% for direct requests, demonstrating superficial filtering mechanisms; Model safety performance varies dramatically from 2\% (claude-opus-4) to 96\% (mistral-small-latest) attack success rates; and eight models exceed 70\% vulnerability when asked to enhance dangerous material properties. We identify fundamental brittleness in current safety alignment, where simple prompt engineering techniques bypass safeguards for dangerous CBRN information. These results challenge industry safety claims and highlight urgent needs for standardized evaluation frameworks, transparent safety metrics, and more robust alignment techniques to mitigate catastrophic misuse risks while preserving beneficial capabilities.

Quantifying CBRN Risk in Frontier Models

TL;DR

This paper addresses the dual-use risk of frontier LLMs leaking CBRN knowledge by conducting an empirical safety evaluation with two complementary datasets: a novel 200-prompt CBRN set and a 180-prompt subset of FORTRESS. It introduces a three-tier attack taxonomy (Direct, Obfuscation, Deep Inception) and uses Attack Success Rate as the primary metric to quantify vulnerability across 10 frontier models. Key findings show substantial safety brittleness, with ASR ranging from 2% to 96% across models and attacks, including dramatic increases under Deep Inception and high susceptibility to enhancement requests. The results reveal significant industry variability in safety implementations and argue for standardized, multi-method evaluation frameworks and deeper alignment techniques to mitigate risks while preserving beneficial capabilities.

Abstract

Frontier Large Language Models (LLMs) pose unprecedented dual-use risks through the potential proliferation of chemical, biological, radiological, and nuclear (CBRN) weapons knowledge. We present the first comprehensive evaluation of 10 leading commercial LLMs against both a novel 200-prompt CBRN dataset and a 180-prompt subset of the FORTRESS benchmark, using a rigorous three-tier attack methodology. Our findings expose critical safety vulnerabilities: Deep Inception attacks achieve 86.0\% success versus 33.8\% for direct requests, demonstrating superficial filtering mechanisms; Model safety performance varies dramatically from 2\% (claude-opus-4) to 96\% (mistral-small-latest) attack success rates; and eight models exceed 70\% vulnerability when asked to enhance dangerous material properties. We identify fundamental brittleness in current safety alignment, where simple prompt engineering techniques bypass safeguards for dangerous CBRN information. These results challenge industry safety claims and highlight urgent needs for standardized evaluation frameworks, transparent safety metrics, and more robust alignment techniques to mitigate catastrophic misuse risks while preserving beneficial capabilities.
Paper Structure (19 sections, 3 figures, 2 tables, 1 algorithm)

This paper contains 19 sections, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Prior CBRN Risk Assessments
  4. AI Safety Frameworks
  5. Methodology and Experimental Setup
  6. Methodology and Experimental Setup
  7. Evaluation Framework and Datasets
  8. Attack Taxonomy and Experimental Protocol
  9. Results
  10. Model and Attack Type Vulnerabilities
  11. Implications of Findings
  12. Ethical Considerations
  13. Discussion
  14. Limitations and Future Work
  15. Limitations.
  16. ...and 4 more sections

Figures (3)

  • Figure 1: CBRN Risk Taxonomy used for dataset creation, showing threat vectors across domains and capability dimensions.
  • Figure 2: Heatmap of Attack Success Rates across models and attack types. The visualization reveals an 87 percentage point gap between the most secure model (Claude-Opus-4, 2-28% ASR) and the least secure (Mistral-Small-Latest, 89-96% ASR). Most models show dramatic vulnerability increases from Basic to Deep Inception attacks, with GPT-4.1 jumping from 27% to 84% ASR and Llama-4-Maverick-Instruct from 17% to 86%.
  • Figure 3: Distribution of Attack Success Rates aggregated across all models by attack type. Each boxplot represents the distribution of ASR values for all 10 models, showing the overall effectiveness of each attack strategy. For the CBRN dataset, median ASR across all models increases from 33.8% (Basic) to 51.5% (Obfuscated) to 86.0% (Deep Inception). Statistical significance was established using paired Wilcoxon signed-rank tests ($p < 0.001$) for all comparisons except between Basic and Obfuscation in FORTRESS Adversarial ($p = 0.78$).