AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applications

TL;DR

The paper tackles the challenge of safely deploying LLM-powered applications by addressing the limitations of manual red-teaming through a scalable, AI-assisted red-teaming pipeline (AART). It presents a four-step, recipe-guided framework that defines evaluation problems, scopes and mixes diverse dimensions (policy concepts, task formats, geographic regions), generates adversarial prompts using an instruction-tuned LLM, and reviews datasets for coverage. Empirical results show AART achieves broader concept coverage and greater linguistic diversity compared to several baselines and automated adapters, while acknowledging limitations of keyword-based evaluation and the need for human expertise for rare, high-risk cases. The work highlights the practical impact of integrating adversarial testing early in product development to reduce risk and improve safety across global, diverse user bases.

Abstract

Adversarial testing of large language models (LLMs) is crucial for their safe and responsible deployment. We introduce a novel approach for automated generation of adversarial evaluation datasets to test the safety of LLM generations on new downstream applications. We call it AI-assisted Red-Teaming (AART) - an automated alternative to current manual red-teaming efforts. AART offers a data generation and augmentation pipeline of reusable and customizable recipes that reduce human effort significantly and enable integration of adversarial testing earlier in new product development. AART generates evaluation datasets with high diversity of content characteristics critical for effective adversarial testing (e.g. sensitive and harmful concepts, specific to a wide range of cultural and geographic regions and application scenarios). The data generation is steered by AI-assisted recipes to define, scope and prioritize diversity within the application context. This feeds into a structured LLM-generation process that scales up evaluation priorities. Compared to some state-of-the-art tools, AART shows promising results in terms of concept coverage and data quality.

Figures (8)

• Figure 1: AI-Assisted Red Teaming supports application teams in developing adversarial datasets with diversity and coverage across multiple dimensions.
• Figure 2: Overview of AI-assisted Red-Teaming approach for automated generation of adversarial datasets
• Figure 3: RealToxicityPrompts
• Figure 4: ParlAI Dialogue Safety
• Figure 5: BAD