Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RedBench: A Universal Dataset for Comprehensive Red Teaming of Large Language Models

Quy-Anh Dang, Chris Ngo, Truong-Son Hy

TL;DR

RedBench tackles fragmentation in red-teaming resources by aggregating 37 benchmarks into 29,362 samples and enforcing a unified taxonomy with 22 risk categories and 19 domains. It employs a semi-automated annotation pipeline to harmonize labels across diverse sources and establishes baselines on six safety-aligned LLMs, revealing that open models are more vulnerable to adversarial prompts while closed models exhibit stronger resistance. The study exposes coverage gaps and imbalances in risk-domain representation, guiding targeted future expansion and more robust benchmarking. All resources and evaluation code are open-source, promoting reproducibility and community-driven advances toward safer, more reliable LLM deployment in safety-critical settings.

Abstract

As large language models (LLMs) become integral to safety-critical applications, ensuring their robustness against adversarial prompts is paramount. However, existing red teaming datasets suffer from inconsistent risk categorizations, limited domain coverage, and outdated evaluations, hindering systematic vulnerability assessments. To address these challenges, we introduce RedBench, a universal dataset aggregating 37 benchmark datasets from leading conferences and repositories, comprising 29,362 samples across attack and refusal prompts. RedBench employs a standardized taxonomy with 22 risk categories and 19 domains, enabling consistent and comprehensive evaluations of LLM vulnerabilities. We provide a detailed analysis of existing datasets, establish baselines for modern LLMs, and open-source the dataset and evaluation code. Our contributions facilitate robust comparisons, foster future research, and promote the development of secure and reliable LLMs for real-world deployment. Code: https://github.com/knoveleng/redeval

RedBench: A Universal Dataset for Comprehensive Red Teaming of Large Language Models

TL;DR

RedBench tackles fragmentation in red-teaming resources by aggregating 37 benchmarks into 29,362 samples and enforcing a unified taxonomy with 22 risk categories and 19 domains. It employs a semi-automated annotation pipeline to harmonize labels across diverse sources and establishes baselines on six safety-aligned LLMs, revealing that open models are more vulnerable to adversarial prompts while closed models exhibit stronger resistance. The study exposes coverage gaps and imbalances in risk-domain representation, guiding targeted future expansion and more robust benchmarking. All resources and evaluation code are open-source, promoting reproducibility and community-driven advances toward safer, more reliable LLM deployment in safety-critical settings.

Abstract

As large language models (LLMs) become integral to safety-critical applications, ensuring their robustness against adversarial prompts is paramount. However, existing red teaming datasets suffer from inconsistent risk categorizations, limited domain coverage, and outdated evaluations, hindering systematic vulnerability assessments. To address these challenges, we introduce RedBench, a universal dataset aggregating 37 benchmark datasets from leading conferences and repositories, comprising 29,362 samples across attack and refusal prompts. RedBench employs a standardized taxonomy with 22 risk categories and 19 domains, enabling consistent and comprehensive evaluations of LLM vulnerabilities. We provide a detailed analysis of existing datasets, establish baselines for modern LLMs, and open-source the dataset and evaluation code. Our contributions facilitate robust comparisons, foster future research, and promote the development of secure and reliable LLMs for real-world deployment. Code: https://github.com/knoveleng/redeval
Paper Structure (32 sections, 2 equations, 7 figures, 12 tables)

This paper contains 32 sections, 2 equations, 7 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Data Collection
  4. Taxonomy for Datasets
  5. Risk Categories
  6. Domains
  7. Annotation Process
  8. Overview of Analysis
  9. Experiments
  10. Metrics
  11. Target LLMs
  12. Results
  13. Results on Refusal Benchmark Datasets
  14. Results on Attack Benchmark Datasets
  15. Conclusion
  16. ...and 17 more sections

Figures (7)

  • Figure 1: Distribution of publication sources for the 37 benchmark datasets in RedBench. The figure illustrates the diversity of high-quality sources, with arXiv, ACL, NeurIPS, and ICLR being the most represented.
  • Figure 2: Heatmap of risk categories by domain for attack prompts in RedBench, illustrating domain-specific risk concentrations.
  • Figure 3: Attack Success Rates (ASR) for the RainbowPlus red teaming method, broken down by risk category and domain, highlighting model-specific vulnerabilities.
  • Figure 4: Rejection Rates of six LLMs across domains for refusal benchmarks, highlighting model-specific and domain-specific over-defensive behaviors.
  • Figure 5: Attack Success Rates for the Direct red teaming method across risk categories and domains.
  • ...and 2 more figures