LLMs Judging LLMs: A Simplex Perspective

TL;DR

This work analyzes the reliability of ranking LLM candidates when evaluations rely on LLM judges without gold-standard labels. It introduces a geometric simplex framework to represent judges and candidates, linking barycentric coordinates to true score prevalences and showing how an augmented simplex height corresponds to expected scores. The authors develop a Bayesian framework with geometric priors to model epistemic uncertainty about judge quality and provide sensitivity analyses across priors, demonstrating that rankings can be robust in some benchmarks but fragile in others. Experiments across multiple datasets show that incorporating epistemic uncertainty yields better calibration (coverage) and often improved ranking accuracy compared to traditional baselines, highlighting the need to account for judge quality in LLM evaluation pipelines.

Abstract

Given the challenge of automatically evaluating free-form outputs from large language models (LLMs), an increasingly common solution is to use LLMs themselves as the judging mechanism, without any gold-standard scores. Implicitly, this practice accounts for only sampling variability (aleatoric uncertainty) and ignores uncertainty about judge quality (epistemic uncertainty). While this is justified if judges are perfectly accurate, it is unclear when such an approach is theoretically valid and practically robust. We study these questions for the task of ranking LLM candidates from a novel geometric perspective: for $M$-level scoring systems, both LLM judges and candidates can be represented as points on an $(M-1)$-dimensional probability simplex, where geometric concepts (e.g., triangle areas) correspond to key ranking concepts. This perspective yields intuitive theoretical conditions and visual proofs for when rankings are identifiable; for instance, we provide a formal basis for the ``folk wisdom'' that LLM judges are more effective for two-level scoring ($M=2$) than multi-level scoring ($M>2$). Leveraging the simplex, we design geometric Bayesian priors that encode epistemic uncertainty about judge quality and vary the priors to conduct sensitivity analyses. Experiments on LLM benchmarks show that rankings based solely on LLM judges are robust in many but not all datasets, underscoring both their widespread success and the need for caution. Our Bayesian method achieves substantially higher coverage rates than existing procedures, highlighting the importance of modeling epistemic uncertainty.

Figures (13)

• Figure 1: LLM judge workflow: For each benchmark question, LLM judges score each candidate's answer according to a rubric. Candidates are ranked based on their judge-assigned scores. Shaded boxes indicate cases where the same LLM serves as both candidate and judge (self-judging).
• Figure 2: The true prevalence of the true scores corresponds to barycentric coordinates; (a) and (b) highlight prevalence $\pi_1$ in 2- and 3-level scoring systems, respectively. A candidate's expected score corresponds to the height of vertical projection in the augmented space, as illustrated in (c).
• Figure 3: Visualization of judge assumptions for the 2-level scoring setting. We suppose there are two LLMs (1=blue, 2=orange), where both are candidates (with score distributions $\gamma_{1}$ and $\gamma_{2}$) and LLM 1 is a judge. (a) Perfect Judge assumes judge vertices $(\theta_{1}, \theta_{2})$ are at the extremes of the 1-dimensional probability simplex. (b) Strong Constancy assumes the vertex positions for the judge are same across all candidates. (c) Moderate Constancy assumes the vertex positions for the judge only differ when self-judging (indicated by the red shifted vertices).
• Figure 4: Non-identifiability in 3-level scoring. Same candidate positions (green/red) explained by different judge configurations yield opposite prevalence rankings: $\pi_{1,1} > \pi_{2,1}$ in (a) vs. $\pi_{1,1} < \pi_{2,1}$ in (b).
• Figure 5: Weight propagation framework for encoding judge quality priors in 3-level scoring, where outgoing edge weights must sum to 1. The probability of the judge assigning score $m_2$ when the true score is $m_1$, $\theta_{m_1, m_2}$, equals the weighted sum of all its parent nodes per the incoming edge weights.

Theorems & Definitions (14)

• Theorem 1
• Theorem 2
• Theorem 3: Height Correspondence in Augmented Simplex
• proof
• proof : Proof for Theorem \ref{['thrm:strong_constancy2']}(i)
• proof : Proof for Theorem \ref{['thrm:strong_constancy2']}(ii)
• Lemma 1
• proof
• Theorem 4: Necessary and Sufficient Conditions
• proof