SAGE: A Generic Framework for LLM Safety Evaluation

TL;DR

SAGE introduces a modular, automated framework for dynamic, multi-turn safety evaluation of LLMs across applications and harm policies. It leverages prompted adversarial user models based on the Big Five to create system-aware conversations, with an evaluator (GPT-4o) and validated human checks to measure policy violations. Across seven state-of-the-art LLMs, SAGE reveals that safety generally declines with longer conversations, model families differ in safety vs. usefulness trade-offs, and policy stringency within a harm category can significantly alter defect rates. The work demonstrates the need for adaptive, policy-aware testing and provides public data and multilingual extensions to encourage ongoing, context-specific safety assessment for real-world deployments.

Abstract

As Large Language Models are rapidly deployed across diverse applications from healthcare to financial advice, safety evaluation struggles to keep pace. Current benchmarks focus on single-turn interactions with generic policies, failing to capture the conversational dynamics of real-world usage and the application-specific harms that emerge in context. Such potential oversights can lead to harms that go unnoticed in standard safety benchmarks and other current evaluation methodologies. To address these needs for robust AI safety evaluation, we introduce SAGE (Safety AI Generic Evaluation), an automated modular framework designed for customized and dynamic harm evaluations. SAGE employs prompted adversarial agents with diverse personalities based on the Big Five model, enabling system-aware multi-turn conversations that adapt to target applications and harm policies. We evaluate seven state-of-the-art LLMs across three applications and harm policies. Multi-turn experiments show that harm increases with conversation length, model behavior varies significantly when exposed to different user personalities and scenarios, and some models minimize harm via high refusal rates that reduce usefulness. We also demonstrate policy sensitivity within a harm category where tightening a child-focused sexual policy substantially increases measured defects across applications. These results motivate adaptive, policy-aware, and context-specific testing for safer real-world deployment.

Figures (5)

• Figure 1: Examples demonstrate how models provide different responses to harmful queries depending on application context, motivating application-specific evaluation.
• Figure 2: Example conversation showing how harmful content can be elicited across multiple turns despite initial refusal. The AI's eventual policy violation would be missed by single-turn evaluation, demonstrating the necessity of conversational safety assessment.
• Figure 3: SAGE framework overview. System Description and Policy Definition (inputs) customize three automated components: Seed Generation produces targeted test topics, User Simulator conducts adversarial conversations with personality-driven agents, and Evaluator judges policy violations. This modular design enables application- and harm-specific safety evaluation at scale.
• Figure 4: Model-level Defect Rates (red) and Refusal Rates (purple) demonstrating the safety-usefulness trade-off. Llama-2 models prioritize safety through high refusal rates, while GPT-4o achieves better balance
• Figure 5: Sample conversations using $\texttt{SAGE}$ for the first two seeds mentioned in Table \ref{['tab:seeds']}. These conversations demonstrate personality-driven attack strategies. (a) Direct, persistent approach (Low Agreeableness, High Extraversion): User repeatedly rephrases harmful requests despite initial refusal, eventually bypassing safety guardrails (Violence policy). (b) Cooperative, build-on-responses approach (High Agreeableness, Low Extraversion): User follows conversational flow naturally, leading the AI to volunteer and reinforce health misinformation without direct prompting (Misinformation policy). Green boxes indicate appropriate refusals; red boxes indicate policy violations.