Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments

Han Zhou, Xingchen Wan, Yinhong Liu, Nigel Collier, Ivan Vulić, Anna Korhonen

TL;DR

LLMs used as evaluators suffer from instruction sensitivity and biased preferences that misalign with human judgments. The authors reveal a strong link between preference fairness and human alignment, and introduce ZEPO, a zero-shot prompt optimization framework that maximizes a fairness objective to steer evaluators toward more human-aligned judgments without labeled data. ZEPO acts as a meta-method, complementary to existing debiasing and calibration techniques, and demonstrates substantial gains across multiple meta-evaluation benchmarks and model families. The work provides a practical, data-free approach to bridge LLM evaluators and human judgments with broad potential for improving automatic evaluation pipelines.

Abstract

Large language models (LLMs) have shown promising abilities as cost-effective and reference-free evaluators for assessing language generation quality. In particular, pairwise LLM evaluators, which compare two generated texts and determine the preferred one, have been employed in a wide range of applications. However, LLMs exhibit preference biases and worrying sensitivity to prompt designs. In this work, we first reveal that the predictive preference of LLMs can be highly brittle and skewed, even with semantically equivalent instructions. We find that fairer predictive preferences from LLMs consistently lead to judgments that are better aligned with humans. Motivated by this phenomenon, we propose an automatic Zero-shot Evaluation-oriented Prompt Optimization framework, ZEPO, which aims to produce fairer preference decisions and improve the alignment of LLM evaluators with human judgments. To this end, we propose a zero-shot learning objective based on the preference decision fairness. ZEPO demonstrates substantial performance improvements over state-of-the-art LLM evaluators, without requiring labeled data, on representative meta-evaluation benchmarks. Our findings underscore the critical correlation between preference fairness and human alignment, positioning ZEPO as an efficient prompt optimizer for bridging the gap between LLM evaluators and human judgments.

Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments

TL;DR

LLMs used as evaluators suffer from instruction sensitivity and biased preferences that misalign with human judgments. The authors reveal a strong link between preference fairness and human alignment, and introduce ZEPO, a zero-shot prompt optimization framework that maximizes a fairness objective to steer evaluators toward more human-aligned judgments without labeled data. ZEPO acts as a meta-method, complementary to existing debiasing and calibration techniques, and demonstrates substantial gains across multiple meta-evaluation benchmarks and model families. The work provides a practical, data-free approach to bridge LLM evaluators and human judgments with broad potential for improving automatic evaluation pipelines.

Abstract

Large language models (LLMs) have shown promising abilities as cost-effective and reference-free evaluators for assessing language generation quality. In particular, pairwise LLM evaluators, which compare two generated texts and determine the preferred one, have been employed in a wide range of applications. However, LLMs exhibit preference biases and worrying sensitivity to prompt designs. In this work, we first reveal that the predictive preference of LLMs can be highly brittle and skewed, even with semantically equivalent instructions. We find that fairer predictive preferences from LLMs consistently lead to judgments that are better aligned with humans. Motivated by this phenomenon, we propose an automatic Zero-shot Evaluation-oriented Prompt Optimization framework, ZEPO, which aims to produce fairer preference decisions and improve the alignment of LLM evaluators with human judgments. To this end, we propose a zero-shot learning objective based on the preference decision fairness. ZEPO demonstrates substantial performance improvements over state-of-the-art LLM evaluators, without requiring labeled data, on representative meta-evaluation benchmarks. Our findings underscore the critical correlation between preference fairness and human alignment, positioning ZEPO as an efficient prompt optimizer for bridging the gap between LLM evaluators and human judgments.
Paper Structure (7 sections, 4 figures, 6 tables, 1 algorithm)

This paper contains 7 sections, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Fairer Preferences Elicit Improved Human-Aligned Judgments
  4. ZEPO: Zero-Shot Prompt Optimization with Fairer Preferences
  5. Experiments and Results
  6. Conclusion
  7. Implementation Details

Figures (4)

  • Figure 1: Illustration of the ZEPO pipeline. Given a manual prompt, the distribution of LLM preferences can be biased towards a certain class. ZEPO optimizes the prompt on a zero-shot fairness learning objective until the balance is achieved in the distribution.
  • Figure 2: LLM evaluators show strong sensitivity to instructions and fairer preference leads to better human-aligned LLM judgments. Sensitivity and evaluation performance studies on preference fairness.
  • Figure 3: Fairness shows the strongest correlation with LLM evaluation performance. Correlation studies of zero-shot learning objectives and LLM evaluation performance. The growth of the x-axis indicates better/stronger fairness, confidence (conf.), and calibration.
  • Figure 4: ZEPO is orthogonal to debiasing approaches and brings further improved LLM judgments. Sensitivity and evaluation performance studies on preference fairness before and after applying permutation debiasing on the COH aspect in SummEval from Llama-3 8B.