Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Are LLM-based Evaluators Confusing NLG Quality Criteria?

Xinyu Hu, Mingqi Gao, Sen Hu, Yang Zhang, Yicheng Chen, Teng Xu, Xiaojun Wan

TL;DR

The paper investigates whether LLM-based evaluators reliably distinguish between NLG quality aspects or whether they confuse criteria. It introduces an 11-aspect hierarchical classification and 18 targeted perturbations, validated with human judgments, to probe LLM evaluation behavior. Across multiple experiments and LLMs, the study finds persistent cross-aspect confusion, sensitivity to criterion descriptions, and limited effectiveness of typical mitigation strategies, even for GPT-4 and specialized Prometheus models. The work highlights fundamental reliability gaps in LLM-based NLG evaluation and provides datasets, perturbations, and evaluation resources to spur further research.

Abstract

Some prior work has shown that LLMs perform well in NLG evaluation for different tasks. However, we discover that LLMs seem to confuse different evaluation criteria, which reduces their reliability. For further verification, we first consider avoiding issues of inconsistent conceptualization and vague expression in existing NLG quality criteria themselves. So we summarize a clear hierarchical classification system for 11 common aspects with corresponding different criteria from previous studies involved. Inspired by behavioral testing, we elaborately design 18 types of aspect-targeted perturbation attacks for fine-grained analysis of the evaluation behaviors of different LLMs. We also conduct human annotations beyond the guidance of the classification system to validate the impact of the perturbations. Our experimental results reveal confusion issues inherent in LLMs, as well as other noteworthy phenomena, and necessitate further research and improvements for LLM-based evaluation.

Are LLM-based Evaluators Confusing NLG Quality Criteria?

TL;DR

The paper investigates whether LLM-based evaluators reliably distinguish between NLG quality aspects or whether they confuse criteria. It introduces an 11-aspect hierarchical classification and 18 targeted perturbations, validated with human judgments, to probe LLM evaluation behavior. Across multiple experiments and LLMs, the study finds persistent cross-aspect confusion, sensitivity to criterion descriptions, and limited effectiveness of typical mitigation strategies, even for GPT-4 and specialized Prometheus models. The work highlights fundamental reliability gaps in LLM-based NLG evaluation and provides datasets, perturbations, and evaluation resources to spur further research.

Abstract

Some prior work has shown that LLMs perform well in NLG evaluation for different tasks. However, we discover that LLMs seem to confuse different evaluation criteria, which reduces their reliability. For further verification, we first consider avoiding issues of inconsistent conceptualization and vague expression in existing NLG quality criteria themselves. So we summarize a clear hierarchical classification system for 11 common aspects with corresponding different criteria from previous studies involved. Inspired by behavioral testing, we elaborately design 18 types of aspect-targeted perturbation attacks for fine-grained analysis of the evaluation behaviors of different LLMs. We also conduct human annotations beyond the guidance of the classification system to validate the impact of the perturbations. Our experimental results reveal confusion issues inherent in LLMs, as well as other noteworthy phenomena, and necessitate further research and improvements for LLM-based evaluation.
Paper Structure (36 sections, 1 equation, 31 figures, 21 tables)

This paper contains 36 sections, 1 equation, 31 figures, 21 tables.

Table of Contents

  1. Introduction
  2. Preliminary Study
  3. Methodology
  4. Classification System for Aspects
  5. Perturbation Attacks
  6. Different Descriptions of Criteria
  7. Data and Test Settings
  8. Datasets.
  9. Datasets.
  10. LLMs in Tests.
  11. Human Judgement
  12. Experiments
  13. Directional Expectation Test
  14. Invariance Test
  15. Different Definition Types
  16. ...and 21 more sections

Figures (31)

  • Figure 1: An example of prompting LLMs to evaluate the dialogue summarization on criteria for fluency.
  • Figure 2: Correlation between scores generated by GPT-3.5 or human annotators on four aspects in SummEval.
  • Figure 3: Our summarized classification system for commonly-used aspects in NLG evaluation and their definitions.
  • Figure 4: Boxplots for items of two tests and correlation matrices for description types of detailed and term.
  • Figure 5: Pearson correlation coefficients between scores generated by GPT-4 or human annotators on four criteria in SummEval.
  • ...and 26 more figures