Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluation

Peng Lai, Zhihao Ou, Yong Wang, Longyue Wang, Jian Yang, Yun Chen, Guanhua Chen

TL;DR

BiasScope presents a fully LLM-driven framework to automatically and at scale discover unknown evaluation biases in LLM-based judges, addressing the limitations of predefined bias lists. It combines an iterative bias discovery and validation loop with a teacher-model perturbation strategy to expand a bias library, and demonstrates that leveraging bias-augmented data can mitigate biases via DPO-based alignment. The framework validates on JudgeBench and yields JudgeBench-Pro, a harder benchmark showing that even powerful LLM evaluators exhibit substantial error rates under bias interference. These findings highlight the need for systematic bias discovery and stronger evaluation robustness as LLM-based judging becomes more prevalent in research and practice.

Abstract

LLM-as-a-Judge has been widely adopted across various research and practical applications, yet the robustness and reliability of its evaluation remain a critical issue. A core challenge it faces is bias, which has primarily been studied in terms of known biases and their impact on evaluation outcomes, while automated and systematic exploration of potential unknown biases is still lacking. Nevertheless, such exploration is crucial for enhancing the robustness and reliability of evaluations. To bridge this gap, we propose BiasScope, a LLM-driven framework for automatically and at scale discovering potential biases that may arise during model evaluation. BiasScope can uncover potential biases across different model families and scales, with its generality and effectiveness validated on the JudgeBench dataset. It overcomes the limitations of existing approaches, transforming bias discovery from a passive process relying on manual effort and predefined bias lists into an active and comprehensive automated exploration. Moreover, based on BiasScope, we propose JudgeBench-Pro, an extended version of JudgeBench and a more challenging benchmark for evaluating the robustness of LLM-as-a-judge. Strikingly, even powerful LLMs as evaluators show error rates above 50\% on JudgeBench-Pro, underscoring the urgent need to strengthen evaluation robustness and to mitigate potential biases further.

BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluation

TL;DR

BiasScope presents a fully LLM-driven framework to automatically and at scale discover unknown evaluation biases in LLM-based judges, addressing the limitations of predefined bias lists. It combines an iterative bias discovery and validation loop with a teacher-model perturbation strategy to expand a bias library, and demonstrates that leveraging bias-augmented data can mitigate biases via DPO-based alignment. The framework validates on JudgeBench and yields JudgeBench-Pro, a harder benchmark showing that even powerful LLM evaluators exhibit substantial error rates under bias interference. These findings highlight the need for systematic bias discovery and stronger evaluation robustness as LLM-based judging becomes more prevalent in research and practice.

Abstract

LLM-as-a-Judge has been widely adopted across various research and practical applications, yet the robustness and reliability of its evaluation remain a critical issue. A core challenge it faces is bias, which has primarily been studied in terms of known biases and their impact on evaluation outcomes, while automated and systematic exploration of potential unknown biases is still lacking. Nevertheless, such exploration is crucial for enhancing the robustness and reliability of evaluations. To bridge this gap, we propose BiasScope, a LLM-driven framework for automatically and at scale discovering potential biases that may arise during model evaluation. BiasScope can uncover potential biases across different model families and scales, with its generality and effectiveness validated on the JudgeBench dataset. It overcomes the limitations of existing approaches, transforming bias discovery from a passive process relying on manual effort and predefined bias lists into an active and comprehensive automated exploration. Moreover, based on BiasScope, we propose JudgeBench-Pro, an extended version of JudgeBench and a more challenging benchmark for evaluating the robustness of LLM-as-a-judge. Strikingly, even powerful LLMs as evaluators show error rates above 50\% on JudgeBench-Pro, underscoring the urgent need to strengthen evaluation robustness and to mitigate potential biases further.
Paper Structure (28 sections, 8 equations, 3 figures, 12 tables, 1 algorithm)

This paper contains 28 sections, 8 equations, 3 figures, 12 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. BiasScope
  3. General Problem Formulation of Automatic Bias Discovery
  4. Efficient Bias Discovery via a Teacher Model
  5. Validating Bias Based on a Test Dataset
  6. Experiments
  7. Experiments Settings
  8. Main Results
  9. Ablation study
  10. In-depth Analysis of BiasScope
  11. Further analysis of BiasScope’s Reliability
  12. Automated Iterations Expand Bias Set Toward Convergence.
  13. Relationship Between Dataset Size and Discovered Biases
  14. From Bias Mining to Mitigation: Alignment with Bias-Augmented Data
  15. Results.
  16. ...and 13 more sections

Figures (3)

  • Figure 1: The Overview of BiasScope. In the Bias Discovery phase (Left), we evaluate the target model on the target dataset perturbed by known biases to expose further potential biases, which are then discovered by a teacher model. In the Bias Validation phase (Right), we introduce a test dataset to examine the effectiveness of the discovered biases. Based on the evaluation results, valid biases are retained and incorporated into the basic bias library to support subsequent iterations.
  • Figure 2: Cumulative Bias Count Across Iterations by Model. Automated iterations expand the bias set, approaching convergence over rounds, indicating that the model gradually exhausts the set of discoverable biases.
  • Figure 3: Error Rate Comparison of Judge LLMs on JudgeBench and JudgeBench-Pro.