Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Aligning Model Evaluations with Human Preferences: Mitigating Token Count Bias in Language Model Assessments

Roland Daynauth, Jason Mars

TL;DR

This paper addresses the misalignment between human preferences and automated evaluators in SLAM's on-device LLM evaluation by identifying token-count bias as a key source of discrepancy. It develops a Bayesian framework and a $t$-test to quantify this bias and introduces a recalibration procedure for GPTScorer to align its judgments with human rankings. Across four use cases, the recalibrated evaluator shows improved correlation with human judgments, including turning the Recommendation use-case from a negative to a strong positive alignment ($-27.27$ to $44.55$). The results demonstrate the feasibility and value of bias correction in automated AI evaluations and provide a framework for extending human-aligned evaluation systems to other biases.

Abstract

The SLAM paper demonstrated that on-device Small Language Models (SLMs) are a viable and cost-effective alternative to API-based Large Language Models (LLMs), such as OpenAI's GPT-4, offering comparable performance and stability. However, SLAM also identified discrepancies between human preferences and traditional auto-evaluators. This follow-up paper explores methods to align LLM evaluator preferences with human evaluations by addressing biases, particularly toward higher token counts. We employed Bayesian statistics and a t-test to quantify this bias and developed a recalibration procedure to adjust the GPTScorer. Our findings significantly improve aligning the recalibrated LLM evaluator with human evaluations across multiple use cases. For instance, spearman's ranking correlation score in the Recommendation use case improved from -27.27 to 44.55. These results highlight the importance of accounting for biases in automated evaluations to ensure fair and accurate model assessments. The recalibration process enhances the reliability of automated evaluators, leading to better AI models that align with human values and expectations. This study provides a robust methodology for future research into bias correction and emphasizes the feasibility and benefits of developing human-aligned AI evaluation systems.

Aligning Model Evaluations with Human Preferences: Mitigating Token Count Bias in Language Model Assessments

TL;DR

This paper addresses the misalignment between human preferences and automated evaluators in SLAM's on-device LLM evaluation by identifying token-count bias as a key source of discrepancy. It develops a Bayesian framework and a -test to quantify this bias and introduces a recalibration procedure for GPTScorer to align its judgments with human rankings. Across four use cases, the recalibrated evaluator shows improved correlation with human judgments, including turning the Recommendation use-case from a negative to a strong positive alignment ( to ). The results demonstrate the feasibility and value of bias correction in automated AI evaluations and provide a framework for extending human-aligned evaluation systems to other biases.

Abstract

The SLAM paper demonstrated that on-device Small Language Models (SLMs) are a viable and cost-effective alternative to API-based Large Language Models (LLMs), such as OpenAI's GPT-4, offering comparable performance and stability. However, SLAM also identified discrepancies between human preferences and traditional auto-evaluators. This follow-up paper explores methods to align LLM evaluator preferences with human evaluations by addressing biases, particularly toward higher token counts. We employed Bayesian statistics and a t-test to quantify this bias and developed a recalibration procedure to adjust the GPTScorer. Our findings significantly improve aligning the recalibrated LLM evaluator with human evaluations across multiple use cases. For instance, spearman's ranking correlation score in the Recommendation use case improved from -27.27 to 44.55. These results highlight the importance of accounting for biases in automated evaluations to ensure fair and accurate model assessments. The recalibration process enhances the reliability of automated evaluators, leading to better AI models that align with human values and expectations. This study provides a robust methodology for future research into bias correction and emphasizes the feasibility and benefits of developing human-aligned AI evaluation systems.
Paper Structure (21 sections, 6 equations, 2 figures, 2 tables)

This paper contains 21 sections, 6 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Objectives
  4. SLAM Evaluation Methodology
  5. Use-Cases Introduction
  6. Human Evaluation
  7. Ranking Human Evaluation
  8. SLAM Auto-Evaluators
  9. Traditional Similarity Evaluators
  10. GPTScorer
  11. Experimental Result
  12. Measuring human token count bias in pairwise evaluation
  13. Measuring Token Bias with Bayesian Statistics
  14. Statistical test for evaluation token bias
  15. Calibrating LLM Evaluation for Token Bias
  16. ...and 6 more sections

Figures (2)

  • Figure 1: Architecture of the SLAM LLM Evaluator
  • Figure 2: Scatter plots showing the alignment between human SLM preference and the length of their output