SLMEval: Entropy-Based Calibration for Human-Aligned Evaluation of Large Language Models

TL;DR

The paper tackles the misalignment of LLM-based evaluators with human judgments in subjective, open-ended tasks. It introduces SLMEval, an entropy-based calibration framework that reweights pairwise evaluator scores using a latent strength distribution $p$ estimated from human data, enforcing robustness via relaxed constraints and solving for maximum entropy. Empirically, SLMEval achieves stronger correlation with human judgments (e.g., $\rho=0.57$) and lower costs (up to $30\x$ savings) than strong baselines like G-Eval and GPT-based scorers across production tasks and open benchmarks. The approach demonstrates practical, scalable evaluation suitable for real-world pipelines, with implications for more reliable model comparisons in subjective domains.

Abstract

The LLM-as-a-Judge paradigm offers a scalable, reference-free approach for evaluating language models. Although several calibration techniques have been proposed to better align these evaluators with human judgment, prior studies focus primarily on narrow, well-structured benchmarks. As a result, it remains unclear whether such calibrations generalize to real-world, open-ended tasks. In this work, we show that SOTA calibrated evaluators often fail in these settings, exhibiting weak or even negative correlation with human judgments. To address this, we propose SLMEval, a novel and efficient calibration method based on entropy maximization over a small amount of human preference data. By estimating a latent distribution over model quality and reweighting evaluator scores accordingly, SLMEval achieves strong correlation with human evaluations across two real-world production use cases and the public benchmark. For example, on one such task, SLMEval achieves a Spearman correlation of 0.57 with human judgments, while G-Eval yields a negative correlation. In addition, SLMEval reduces evaluation costs by 5-30x compared to GPT-4-based calibrated evaluators such as G-eval.