Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusal

Haonan Zhang, Dongxia Wang, Yi Liu, Kexin Chen, Jiashui Wang, Xinlei Ying, Long Liu, Wenhai Wang

TL;DR

This work tackles the problem of over-refusal, where LLMs reject benign prompts due to overly cautious safety. It introduces ORFuzz, an evolutionary fuzzing framework combining category-aware seed selection, adaptive mutator optimization, and OR-Judge, a human-aligned validator, to systematically uncover over-refusal vulnerabilities. Through a motivating user study, ORFuzz demonstrates superior generation of valid over-refusal instances (average ORR 6.98%) and yields ORFuzzSet, a transferable benchmark of 1,786 prompts achieving 57.37% average ORR across 14 LLMs. The framework’s components are shown to be complementary and essential, enabling targeted fine-tuning and a real-time refusal checker, with broad implications for building more trustworthy LLM systems.

Abstract

Large Language Models (LLMs) increasingly exhibit over-refusal - erroneously rejecting benign queries due to overly conservative safety measures - a critical functional flaw that undermines their reliability and usability. Current methods for testing this behavior are demonstrably inadequate, suffering from flawed benchmarks and limited test generation capabilities, as highlighted by our empirical user study. To the best of our knowledge, this paper introduces the first evolutionary testing framework, ORFuzz, for the systematic detection and analysis of LLM over-refusals. ORFuzz uniquely integrates three core components: (1) safety category-aware seed selection for comprehensive test coverage, (2) adaptive mutator optimization using reasoning LLMs to generate effective test cases, and (3) OR-Judge, a human-aligned judge model validated to accurately reflect user perception of toxicity and refusal. Our extensive evaluations demonstrate that ORFuzz generates diverse, validated over-refusal instances at a rate (6.98% average) more than double that of leading baselines, effectively uncovering vulnerabilities. Furthermore, ORFuzz's outputs form the basis of ORFuzzSet, a new benchmark of 1,855 highly transferable test cases that achieves a superior 63.56% average over-refusal rate across 10 diverse LLMs, significantly outperforming existing datasets. ORFuzz and ORFuzzSet provide a robust automated testing framework and a valuable community resource, paving the way for developing more reliable and trustworthy LLM-based software systems.

ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusal

TL;DR

This work tackles the problem of over-refusal, where LLMs reject benign prompts due to overly cautious safety. It introduces ORFuzz, an evolutionary fuzzing framework combining category-aware seed selection, adaptive mutator optimization, and OR-Judge, a human-aligned validator, to systematically uncover over-refusal vulnerabilities. Through a motivating user study, ORFuzz demonstrates superior generation of valid over-refusal instances (average ORR 6.98%) and yields ORFuzzSet, a transferable benchmark of 1,786 prompts achieving 57.37% average ORR across 14 LLMs. The framework’s components are shown to be complementary and essential, enabling targeted fine-tuning and a real-time refusal checker, with broad implications for building more trustworthy LLM systems.

Abstract

Large Language Models (LLMs) increasingly exhibit over-refusal - erroneously rejecting benign queries due to overly conservative safety measures - a critical functional flaw that undermines their reliability and usability. Current methods for testing this behavior are demonstrably inadequate, suffering from flawed benchmarks and limited test generation capabilities, as highlighted by our empirical user study. To the best of our knowledge, this paper introduces the first evolutionary testing framework, ORFuzz, for the systematic detection and analysis of LLM over-refusals. ORFuzz uniquely integrates three core components: (1) safety category-aware seed selection for comprehensive test coverage, (2) adaptive mutator optimization using reasoning LLMs to generate effective test cases, and (3) OR-Judge, a human-aligned judge model validated to accurately reflect user perception of toxicity and refusal. Our extensive evaluations demonstrate that ORFuzz generates diverse, validated over-refusal instances at a rate (6.98% average) more than double that of leading baselines, effectively uncovering vulnerabilities. Furthermore, ORFuzz's outputs form the basis of ORFuzzSet, a new benchmark of 1,855 highly transferable test cases that achieves a superior 63.56% average over-refusal rate across 10 diverse LLMs, significantly outperforming existing datasets. ORFuzz and ORFuzzSet provide a robust automated testing framework and a valuable community resource, paving the way for developing more reliable and trustworthy LLM-based software systems.

Paper Structure

This paper contains 40 sections, 9 equations, 4 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM Jailbreaking and Defenses
  4. Over-Refusal Behavior in LLMs
  5. Fuzzing Techniques for LLMs
  6. Motivating User Study
  7. Study Design and Setup
  8. Evaluation Metrics and Data Analysis
  9. Key Findings and Observations
  10. Motivations for a New Testing Framework
  11. ORFuzz
  12. Category-Aware Seed Selection
  13. Adaptive Mutator Selection and Refinement
  14. Mutator Design
  15. Adaptive Mutator Selection and Refinement
  16. ...and 25 more sections

Figures (4)

  • Figure 1: The security duality: LLMs strive to block harmful content but often over-censor benign queries, leading to over-refusal (a type of functional failure).
  • Figure 2: The workflow of ORFuzz.
  • Figure 3: Transferability heatmap of ORFuzz-generated over-refusal samples. Rows: source LLM (samples generated for); Columns: target LLM (samples evaluated on). Cell values: Over-Refusal Rate (ORR).
  • Figure 4: Left: ORR of new benchmark ORFuzzSet vs. existing datasets on 14 LLMs. Right: t-SNE of ORFuzzSet's semantic diversity.