CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models

TL;DR

The paper tackles the problem that existing LLM safety benchmarks largely ignore the surrounding context of queries, which can lead to over-refusal and poor user experience. It introduces CASE-Bench, a Context-Aware SafEty Benchmark that ties each query to formalized CI-based contexts (Sender, Recipient, Transmission Principle) and collects non-binary safety judgments from 2,000+ annotators across 900 query-context pairs. The authors demonstrate a substantial and statistically significant influence of context on human safety judgments ($p<0.0001$) and reveal notable mismatches between human judgments and several commercial models, highlighting challenges in current safety moderation. The work provides a rigorous data-collection and analysis pipeline (power analysis to determine annotator numbers, CI-parameter adaptations, automatic then manual context generation, and thorough annotation) and shows how context-aware evaluation can more accurately assess LLM safety and guide safer deployment in real-world use. The results establish CASE-Bench as a foundation for future context-integrated safety evaluation and offer design insights for mitigating over-refusal and improving alignment with human judgments.

Abstract

Aligning large language models (LLMs) with human values is essential for their safe deployment and widespread adoption. Current LLM safety benchmarks often focus solely on the refusal of individual problematic queries, which overlooks the importance of the context where the query occurs and may cause undesired refusal of queries under safe contexts that diminish user experience. Addressing this gap, we introduce CASE-Bench, a Context-Aware SafEty Benchmark that integrates context into safety assessments of LLMs. CASE-Bench assigns distinct, formally described contexts to categorized queries based on Contextual Integrity theory. Additionally, in contrast to previous studies which mainly rely on majority voting from just a few annotators, we recruited a sufficient number of annotators necessary to ensure the detection of statistically significant differences among the experimental conditions based on power analysis. Our extensive analysis using CASE-Bench on various open-source and commercial LLMs reveals a substantial and significant influence of context on human judgments (p<0.0001 from a z-test), underscoring the necessity of context in safety evaluations. We also identify notable mismatches between human judgments and LLM responses, particularly in commercial models within safe contexts.

Figures (10)

• Figure 1: Visualization of an example where context influences human judgments on whether it is safe to respond to a user's query. Context is formalized using CI parameters: sender, recipient, and transmission principle. Left: No context; Middle: Safe context; Right: Unsafe context. More context examples can be found in the tutorial in Appendix \ref{['Appendix: tutorial']}.
• Figure 2: Data creation pipeline for CASE-Bench which sequentially executes query selection, automatic context generation and manual revision stages. Annotators are involved in each stage.
• Figure 3: Visualization of Kruskal-Wallis test results across 45 categories in the CASE-Bench dataset. The chart distinguishes between significant and non-significant categories, with bars representing the average K-W statistic values. Categories labelled as "Non-significant" are displayed with a hatched pattern. Category "Child-related Crimes" is excluded as detailed in §\ref{['sec:query-selection']}.
• Figure 4: Plot of correlation between LLM safety judgements and safety ratings given by the crowd of annotators. As multiple dots may overlay on each other, the density of the dots is also indicated by the colour map in the background where darker means denser.
• Figure 5: Recall rates for safe and unsafe contexts with different subsets of CI parameters for Llama-3 (upper) and GPT-4o-mini (middle) and Claude-3.5 (bottom). S denotes sender, R denotes recipient, TP denotes the transmission principle.