Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Finding Blind Spots in Evaluator LLMs with Interpretable Checklists

Sumanth Doddapaneni, Mohammed Safi Ur Rahman Khan, Sshubam Verma, Mitesh M. Khapra

TL;DR

FBI is proposed, a novel framework designed to examine the proficiency of Evaluator LLMs in assessing four critical abilities in other LLMs: factual accuracy, instruction following, coherence in long-form writing, and reasoning proficiency, by introducing targeted perturbations in answers generated by LLMs.

Abstract

Large Language Models (LLMs) are increasingly relied upon to evaluate text outputs of other LLMs, thereby influencing leaderboards and development decisions. However, concerns persist over the accuracy of these assessments and the potential for misleading conclusions. In this work, we investigate the effectiveness of LLMs as evaluators for text generation tasks. We propose FBI, a novel framework designed to examine the proficiency of Evaluator LLMs in assessing four critical abilities in other LLMs: factual accuracy, instruction following, coherence in long-form writing, and reasoning proficiency. By introducing targeted perturbations in answers generated by LLMs, that clearly impact one of these key capabilities, we test whether an Evaluator LLM can detect these quality drops. By creating a total of 2400 perturbed answers covering 22 perturbation categories, we conduct a comprehensive study using different evaluation strategies on five prominent LLMs commonly used as evaluators in the literature. Our findings reveal significant shortcomings in current Evaluator LLMs, which failed to identify quality drops in over 50\% of cases on average. Single-answer and pairwise evaluations demonstrated notable limitations, whereas reference-based evaluations showed comparatively better performance. These results underscore the unreliable nature of current Evaluator LLMs and advocate for cautious implementation in practical applications. Code and data are available at https://github.com/AI4Bharat/FBI.

Finding Blind Spots in Evaluator LLMs with Interpretable Checklists

TL;DR

FBI is proposed, a novel framework designed to examine the proficiency of Evaluator LLMs in assessing four critical abilities in other LLMs: factual accuracy, instruction following, coherence in long-form writing, and reasoning proficiency, by introducing targeted perturbations in answers generated by LLMs.

Abstract

Large Language Models (LLMs) are increasingly relied upon to evaluate text outputs of other LLMs, thereby influencing leaderboards and development decisions. However, concerns persist over the accuracy of these assessments and the potential for misleading conclusions. In this work, we investigate the effectiveness of LLMs as evaluators for text generation tasks. We propose FBI, a novel framework designed to examine the proficiency of Evaluator LLMs in assessing four critical abilities in other LLMs: factual accuracy, instruction following, coherence in long-form writing, and reasoning proficiency. By introducing targeted perturbations in answers generated by LLMs, that clearly impact one of these key capabilities, we test whether an Evaluator LLM can detect these quality drops. By creating a total of 2400 perturbed answers covering 22 perturbation categories, we conduct a comprehensive study using different evaluation strategies on five prominent LLMs commonly used as evaluators in the literature. Our findings reveal significant shortcomings in current Evaluator LLMs, which failed to identify quality drops in over 50\% of cases on average. Single-answer and pairwise evaluations demonstrated notable limitations, whereas reference-based evaluations showed comparatively better performance. These results underscore the unreliable nature of current Evaluator LLMs and advocate for cautious implementation in practical applications. Code and data are available at https://github.com/AI4Bharat/FBI.
Paper Structure (34 sections, 3 figures, 24 tables)

This paper contains 34 sections, 3 figures, 24 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. FBI: Meta-Evaluation Checklist
  4. Prompt Selection
  5. Perturbation Categories
  6. Perturbation Generation
  7. Human-In-The-Loop
  8. Score-Invariant Perturbations
  9. Strategies for using Evaluator LLMs
  10. Single Answer Scoring
  11. Vanilla$^*$
  12. Vanilla
  13. Rubric
  14. Axis
  15. Axis+Rubric
  16. ...and 19 more sections

Figures (3)

  • Figure 1: We present FBI, our novel meta-evaluation framework designed to assess the robustness of evaluator LLMs across diverse tasks and evaluation strategies.
  • Figure 2: Comparison of perturbations detected solely by score analysis versus those identified with explanations. The highlighted region marked with stars denotes perturbations detected in explanations but not reflected in scores. Despite this, a significant proportion of perturbations remain undetected.
  • Figure 3: Screenshot of the User Application developed for validating perturbations.