FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge

TL;DR

FairJudge treats judging as a learnable policy to address position bias, non-semantic biases, and cross-mode inconsistencies that plague LLM-based evaluators. It introduces a high-information-density judging dataset (FairJudge-16K) and a curriculum training pipeline (SFT-DPO-GRPO) to achieve rubric-aware adaptivity, debiasing, and cross-mode consistency, validated on PandaLM, JudgeLM, and multimodal benchmarks. Across ablations and baselines, FairJudge demonstrates stronger agreement and macro-F1, robust cross-mode performance, and efficient inference, highlighting the importance of learning judging behavior rather than relying on static prompts or scale alone. The approach and released data aim to improve the reliability, fairness, and reproducibility of automated evaluation in real-world ML development and research contexts.

Abstract

Existing LLM-as-a-Judge systems suffer from three fundamental limitations: limited adaptivity to task- and domain-specific evaluation criteria, systematic biases driven by non-semantic cues such as position, length, format, and model provenance, and evaluation inconsistency that leads to contradictory judgments across different evaluation modes (e.g., pointwise versus pairwise). To address these issues, we propose FairJudge, an adaptive, debiased, and consistent LLM-as-a-Judge. Unlike prior approaches that treat the judge as a static evaluator, FairJudge models judging behavior itself as a learnable and regularized policy. From a data-centric perspective, we construct a high-information-density judging dataset that explicitly injects supervision signals aligned with evaluation behavior. Building on this dataset, we adopt a curriculum-style SFT-DPO-GRPO training paradigm that progressively aligns rubric adherence, bias mitigation, and cross-mode consistency, while avoiding catastrophic forgetting. Experimental results on multiple internal and public benchmarks show that FairJudge consistently improves agreement and F1, reduces non-semantic biases, and outperforms substantially larger instruction-tuned LLMs. All resources will be publicly released after acceptance to facilitate future research.

Figures (11)

• Figure 1: Motivation---Two representative issues in LLM-as-a-Judge. Top: Position bias, where the judgment flips when the order of answers is swapped. Bottom: Pointwise--pairwise inconsistency, where the same answers receive contradictory judgments under different evaluation modes. Representative real-world examples are provided in Appendix \ref{['real bad case']}.
• Figure 2: Data construction pipeline of FairJudge. Left (SFT data): Source evaluation records are organized into {Task, Reference, Answer Pair} and augmented with explicit {Rubric, Reasoning, Judgment}. The rubric is used both as a conditioning input and, in selected cases, as a prediction target. Middle (DPO data): Preference pairs (chosen/rejected) are constructed on the same evaluation instance under targeted non-semantic perturbations, guiding the judge to be robust to non-semantic biases. Right (GRPO data): Cross-mode samples are organized by jointly constructing pointwise (scores or labels) and pairwise (relative preference) evaluations for the same instance, with consistency rewards aligning judgments across modes to enforce cross-mode consistency.
• Figure 3: Training pipeline of FairJudge. The model is first trained with SFT data, where rubrics are used as conditioning inputs and, in some cases, as prediction targets, enabling explicit modeling of evaluation criteria. It is then optimized with DPO data in the form of chosen/rejected judgment pairs to reduce non-semantic biases. Finally, GRPO training applies consistency-oriented rewards, encouraging judgments that remain stable across evaluation settings. This staged process yields a rubric-aware, debiased, and consistent judge.
• Figure 4: Comparison between function-based judging and policy-based judging.
• Figure 5: Representative data formats used in different training stages of FairJudge, including distilled judgment examples for supervised fine-tuning (SFT) (top: pair-wise and point-wise), as well as training data for preference alignment and consistency optimization via DPO and GRPO (bottom).