Bayesian Calibration of Win Rate Estimation with LLM Evaluators

TL;DR

The paper tackles the bias that arises when using LLM evaluators to compare generative models, formalizing true win rate $p$ and observed win rate $k_e$ with evaluator accuracies $q_0^e$ and $q_1^e$. It introduces two Bayesian calibration methods, Bayesian Win Rate Sampling (BWRS) and Bayesian Dawid-Skene, to infer $p$ from evaluator judgments and optionally limited human data, using Beta-Bernoulli posteriors and HMC/NUTS inference. Empirically, it validates the approaches on six datasets spanning story generation, summarization, and instruction following, showing reduced estimation bias relative to the naive $k_e$ baseline, with the gains amplified when informative priors are available. The work supports more reliable automatic text quality evaluation and highlights practical considerations such as evaluator accuracy, prior data, and the need for robust annotation models in LLM-based assessment pipelines.

Abstract

Recent advances in large language models (LLMs) show the potential of using LLMs as evaluators for assessing the quality of text generations from LLMs. However, applying LLM evaluators naively to compare or judge between different systems can lead to unreliable results due to the intrinsic win rate estimation bias of LLM evaluators. In order to mitigate this problem, we propose two calibration methods, Bayesian Win Rate Sampling (BWRS) and Bayesian Dawid-Skene, both of which leverage Bayesian inference to more accurately infer the true win rate of generative language models. We empirically validate our methods on six datasets covering story generation, summarization, and instruction following tasks. We show that both our methods are effective in improving the accuracy of win rate estimation using LLMs as evaluators, offering a promising direction for reliable automatic text quality evaluation.

Figures (2)

• Figure 1: Illustration of our pipeline and previous work. The "calibration" part of our pipeline indicates one of BWRS or Bayesian Dawid-Skene.
• Figure 2: Win rate estimation error with various proportions of the original data used as in-distribution prior. The results are averaged over all compared generator pairs. The mean and variance of all results are calculated over ten repetitive runs. The variance of $k$ values in the three instruction following datasets results from randomly assigning outputs to two simulative generators, as described in Section \ref{['subsec:datasets']}

Theorems & Definitions (2)

• Definition 1: True win rate
• Definition 2: Observed win rate