Leveraging LLMs as Meta-Judges: A Multi-Agent Framework for Evaluating LLM Judgments

TL;DR

This work tackles the challenge of evaluating LLM judgments by proposing a meta-judge pipeline that uses a rubric refined by GPT-4 and human experts, a multi-agent module to score judgments, and a threshold-based selection to filter reliable judgments. Unlike prior work that relies on single-model judgments or human alignment, it jointly addresses biases in human evaluation and the reliability of LLM judgments. The authors systematically compare single-agent and multi-agent configurations on JudgeBench, showing meaningful precision gains over raw judgments and single-agent baselines. The results suggest LLMs can serve as meta-judges to curate high-quality judgments and lay groundwork for automatic construction of preference datasets to train LLM-based evaluators.

Abstract

Large language models (LLMs) are being widely applied across various fields, but as tasks become more complex, evaluating their responses is increasingly challenging. Compared to human evaluators, the use of LLMs to support performance evaluation offers a more efficient alternative. However, most studies focus mainly on aligning LLMs' judgments with human preferences, overlooking the existence of biases and mistakes in human judgment. Furthermore, how to select suitable LLM judgments given multiple potential LLM responses remains underexplored. To address these two aforementioned issues, we propose a three-stage meta-judge selection pipeline: 1) developing a comprehensive rubric with GPT-4 and human experts, 2) using three advanced LLM agents to score judgments, and 3) applying a threshold to filter out low-scoring judgments. Compared to methods using a single LLM as both judge and meta-judge, our pipeline introduces multi-agent collaboration and a more comprehensive rubric. Experimental results on the JudgeBench dataset show about 15.55\% improvement compared to raw judgments and about 8.37\% improvement over the single-agent baseline. Our work demonstrates the potential of LLMs as meta-judges and lays the foundation for future research on constructing preference datasets for LLM-as-a-judge reinforcement learning.

Figures (5)

• Figure 1: Method components and interactions. The actor (an LLM) generates responses, the judge evaluates their quality, and the meta-judge assesses the judge’s evaluation. In the diagram, the backward RL (DPO) arrow represents Reinforcement Learning from AI Feedback (RLAIF), using DPO as the training method.
• Figure 2: LLM-as-meta-judge framework. The rubric is predefined in the prompt design stage. We benchmark the judgment using N agents, each providing a score based on a rubric. These N scores are then aggregated through metric calculations to yield a comprehensive score reflecting the LLM judge's performance.
• Figure 3: Cooperative discussion diagram. Each agent is assigned a distinct role. Agent 1 shares its meta-judgment with Agent 2, which updates its own judgment accordingly. Agent 3 then refines its meta-judgment by integrating inputs from Agents 1 and 2. An additional Agent 4 can be applied to summarize all meta-judgments to produce the final outcome.
• Figure 4: Generation of Raw Judgments from the JudgeBench Dataset. A judgment is considered true if it matches the label for the answer pairs; otherwise, it is false.
• Figure 5: Ablation study on the impact of the number and roles of agents across different tasks. The vertical axis represents the precision of the selected judgments by multi-agent meta-judging. The horizontal axis represents different tasks and the overall summary of all tasks.