Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nonparametric LLM Evaluation from Preference Data

Dennis Frauen, Athiya Deviyani, Mihaela van der Schaar, Stefan Feuerriegel

TL;DR

The paper tackles the challenge of ranking LLMs from context-dependent human preferences by proposing a nonparametric framework (DMLEval / DMLRank) built around generalized average ranking scores (GARS), which express the target ranking as $\theta=\mathbb{E}[F(\mu(X))]$. It develops a debiased, EIF-based estimator that remains valid when nuisance components are estimated with black-box models, and it integrates external judges and budget-aware data collection. The framework unifies multiple ranking notions (e.g., Borda, Bradley–Terry, Rank Centrality) under one target and provides an explicit A-optimal data-acquisition policy to minimize estimator variance under cost. Empirically, it demonstrates improved statistical efficiency and principled uncertainty quantification on synthetic data and real-world LLM preference datasets, with robust rankings that are less sensitive to BT misspecification. This approach advances practical LLM leaderboards by enabling flexible, reliable, and cost-effective evaluation from preference data.

Abstract

Evaluating the performance of large language models (LLMs) from human preference data is crucial for obtaining LLM leaderboards. However, many existing approaches either rely on restrictive parametric assumptions or lack valid uncertainty quantification when flexible machine learning methods are used. In this paper, we propose a nonparametric statistical framework, DMLEval, for comparing and ranking LLMs from preference data using debiased machine learning (DML). For this, we introduce generalized average ranking scores (GARS), which generalize commonly used ranking models, including the Bradley-Terry model or PageRank/ Rank centrality, with complex human responses such as ties. DMLEval comes with the following advantages: (i) It produces statistically efficient estimates of GARS ranking scores. (ii) It naturally allows the incorporation of black-box machine learning methods for estimation. (iii) It can be combined with pre-trained LLM evaluators (e.g., using LLM-as-a-judge). (iv) It suggests optimal policies for collecting preference data under budget constraints. We demonstrate these advantages both theoretically and empirically using both synthetic and real-world preference datasets. In summary, our framework provides practitioners with powerful, state-of-the-art methods for comparing or ranking LLMs.

Nonparametric LLM Evaluation from Preference Data

TL;DR

The paper tackles the challenge of ranking LLMs from context-dependent human preferences by proposing a nonparametric framework (DMLEval / DMLRank) built around generalized average ranking scores (GARS), which express the target ranking as . It develops a debiased, EIF-based estimator that remains valid when nuisance components are estimated with black-box models, and it integrates external judges and budget-aware data collection. The framework unifies multiple ranking notions (e.g., Borda, Bradley–Terry, Rank Centrality) under one target and provides an explicit A-optimal data-acquisition policy to minimize estimator variance under cost. Empirically, it demonstrates improved statistical efficiency and principled uncertainty quantification on synthetic data and real-world LLM preference datasets, with robust rankings that are less sensitive to BT misspecification. This approach advances practical LLM leaderboards by enabling flexible, reliable, and cost-effective evaluation from preference data.

Abstract

Evaluating the performance of large language models (LLMs) from human preference data is crucial for obtaining LLM leaderboards. However, many existing approaches either rely on restrictive parametric assumptions or lack valid uncertainty quantification when flexible machine learning methods are used. In this paper, we propose a nonparametric statistical framework, DMLEval, for comparing and ranking LLMs from preference data using debiased machine learning (DML). For this, we introduce generalized average ranking scores (GARS), which generalize commonly used ranking models, including the Bradley-Terry model or PageRank/ Rank centrality, with complex human responses such as ties. DMLEval comes with the following advantages: (i) It produces statistically efficient estimates of GARS ranking scores. (ii) It naturally allows the incorporation of black-box machine learning methods for estimation. (iii) It can be combined with pre-trained LLM evaluators (e.g., using LLM-as-a-judge). (iv) It suggests optimal policies for collecting preference data under budget constraints. We demonstrate these advantages both theoretically and empirically using both synthetic and real-world preference datasets. In summary, our framework provides practitioners with powerful, state-of-the-art methods for comparing or ranking LLMs.
Paper Structure (81 sections, 6 theorems, 138 equations, 5 figures, 6 tables)

This paper contains 81 sections, 6 theorems, 138 equations, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Problem setting
  4. Generalized Average Ranking Scores
  5. Motivation
  6. Formulation of GARS
  7. Efficient statistical inference
  8. Overview
  9. Main result
  10. Implementation
  11. Optimal preference data acquisition
  12. Experiments
  13. Synthetic data
  14. Real-world data
  15. Extended related work on experimental design
  16. ...and 66 more sections

Key Result

Theorem 5.1

Let $F:[0,1]^{K \times K \times C}\to\mathbb R^d$ be differentiable with Jacobian columns $J_{jk}(\mu)\;=\;\nabla_{\mu_{jk}} F(\mu)\;\in\;\mathbb R^{d\times C}$. The efficient influence function (EIF) for $\theta$ is The (one-step) debiased estimator $\hat{\theta}_\mathrm{EIF} \in\mathbb R^d$ is given via Under standard assumptions (see Appendix app:proofs for details), $\hat{\theta}_\mathrm{EIF

Figures (5)

  • Figure 1: Overview of preference-based LLM evaluation.
  • Figure 2: Overview of statistical inference for GARS.
  • Figure 3: Results for Chatbot Arena preference data. Shown: estimated ranking scores for different GARS functionals (Borda, BT, and rank centrality) and estimators (debiased and plugin). Changes in ranking are indicated in red and green (for debiased estimators as compared to debiased Borda scores, for plugin estimators as compared to the corresponding debiased estimator). We report 95% simultaneous confidence intervals, which attain near-zero width for plugin estimators.
  • Figure 4: Synthetic experimental results. (a) Relative ranking error normalized by the no-judge baseline (mean and $95\%$ CIs over $n=100$ runs). (b) BT-projection estimation error (mean and $95\%$ CIs over $30$ runs).
  • Figure 5: Results for MT-Bench preference data. Shown: estimated ranking scores for different GARS functionals (Borda, BT, and rank centrality) and estimators (debiased and plugin). Changes in ranking are indicated in red and green (for debiased estimators as compared to debiased Borda scores, for plugin estimators as compared to the corresponding debiased estimator). We report 95% simultaneous confidence intervals, which attain near-zero width for plugin estimators.

Theorems & Definitions (14)

  • Theorem 5.1: Efficient statistical inference for GARS
  • proof
  • Definition 6.1
  • Theorem 6.2: A-optimal labeling policy
  • proof
  • proof
  • proof
  • Theorem 5.1: EIF under the BT restriction
  • Corollary 5.2: A-optimal policy under BT restriction
  • Definition 6.2: A-optimality under at-most-one selection
  • ...and 4 more