Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dropping Just a Handful of Preferences Can Change Top Large Language Model Rankings

Jenny Y. Huang, Yunyi Shen, Dennis Wei, Tamara Broderick

TL;DR

The paper addresses the fragility of BT-based LLM leaderboards to dropping a vanishingly small fraction of preference data. It extends AMIP-style robustness ideas to ranking, proposing a fast, pairwise-robustness framework that identifies influential data points and then verifies their impact by refitting BT models. Empirically, it shows that top-ranked models can flip with as little as $0.003\%$ of data on popular arenas (notably Chatbot Arena), while MT-bench remains comparatively robust, likely due to expert annotations. The work highlights practical implications for leaderboard design and calls for robustness checks and design improvements to reduce fragility and improve reliability in AI benchmarking.

Abstract

We propose a method for evaluating the robustness of widely used LLM ranking systems -- variants of a Bradley--Terry model -- to dropping a worst-case very small fraction of preference data. Our approach is computationally fast and easy to adopt. When we apply our method to matchups from popular LLM ranking platforms, including Chatbot Arena and derivatives, we find that the rankings of top-performing models can be remarkably sensitive to the removal of a small fraction of preferences; for instance, dropping just 0.003% of human preferences can change the top-ranked model on Chatbot Arena. Our robustness check identifies the specific preferences most responsible for such ranking flips, allowing for inspection of these influential preferences. We observe that the rankings derived from MT-bench preferences are notably more robust than those from Chatbot Arena, likely due to MT-bench's use of expert annotators and carefully constructed prompts. Finally, we find that neither rankings based on crowdsourced human evaluations nor those based on LLM-as-a-judge preferences are systematically more sensitive than the other.

Dropping Just a Handful of Preferences Can Change Top Large Language Model Rankings

TL;DR

The paper addresses the fragility of BT-based LLM leaderboards to dropping a vanishingly small fraction of preference data. It extends AMIP-style robustness ideas to ranking, proposing a fast, pairwise-robustness framework that identifies influential data points and then verifies their impact by refitting BT models. Empirically, it shows that top-ranked models can flip with as little as of data on popular arenas (notably Chatbot Arena), while MT-bench remains comparatively robust, likely due to expert annotations. The work highlights practical implications for leaderboard design and calls for robustness checks and design improvements to reduce fragility and improve reliability in AI benchmarking.

Abstract

We propose a method for evaluating the robustness of widely used LLM ranking systems -- variants of a Bradley--Terry model -- to dropping a worst-case very small fraction of preference data. Our approach is computationally fast and easy to adopt. When we apply our method to matchups from popular LLM ranking platforms, including Chatbot Arena and derivatives, we find that the rankings of top-performing models can be remarkably sensitive to the removal of a small fraction of preferences; for instance, dropping just 0.003% of human preferences can change the top-ranked model on Chatbot Arena. Our robustness check identifies the specific preferences most responsible for such ranking flips, allowing for inspection of these influential preferences. We observe that the rankings derived from MT-bench preferences are notably more robust than those from Chatbot Arena, likely due to MT-bench's use of expert annotators and carefully constructed prompts. Finally, we find that neither rankings based on crowdsourced human evaluations nor those based on LLM-as-a-judge preferences are systematically more sensitive than the other.

Paper Structure

This paper contains 43 sections, 1 theorem, 21 equations, 18 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Setup
  3. Human preference data.
  4. Proposed method
  5. Runtime.
  6. Experiments
  7. Data and Setup
  8. Sensitivity of LLM Arenas
  9. Humans vs. LLM-as-a-Judge
  10. Inspecting Dropped Preferences
  11. Related Work
  12. Vulnerabilities in AI Leaderboards
  13. Data-dropping Robustness
  14. Discussion
  15. Uncertainty Quantification
  16. ...and 28 more sections

Key Result

Proposition B.1

Suppose we have $M$ real numbers, $\mathcal{T}(w):=\{\widehat{\theta}_{i}(w)\}_{i=1}^M$. Suppose a set $\mathcal{S} \subset \mathcal{T}(w)$ satisfies $|\mathcal{S}|=k$. Suppose it is the case that $\forall \; \widehat{\theta}_{i}(w) \in \mathcal{S}$ and $\forall \; \widehat{\theta}_{j}(w) \in \mathc

Figures (18)

  • Figure 1: Our method (i) tests whether AI leaderboard rankings remain stable upon dropping small fractions of data and (ii) pinpoints the specific data points (e.g., preferences) that drive ranking flips.
  • Figure 2: Each bar shows the fraction of data points dropped from Chatbot Arena that is sufficient to demote the BT score of a model inside the top-$k$ to outside of the top-$k$ ($k \in \{1, 3, 5, 10, 20\}$). The orange bars correspond to human evaluators and green bars to LLM-as-a-judge evaluators.
  • Figure 3: Bootstrap-confidence-interval-based rankings on Chatbot Arena (Human Judge).
  • Figure 4: Bootstrap-confidence-interval-based rankings on Chatbot Arena (LLM Judge).
  • Figure 5: Bootstrap-confidence-interval-based rankings on Vision Arena.
  • ...and 13 more figures

Theorems & Definitions (5)

  • Definition 1: Feasible Drop Set
  • Definition 2: Top-$k$ Set
  • Definition 3: Top-$k$ Data-Dropping Robustness
  • Proposition B.1
  • proof