Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Finetuning LLMs for Comparative Assessment Tasks

Vatsal Raina, Adian Liusie, Mark Gales

TL;DR

This work proposes a framework for finetuning LLMs for comparative assessment to align the model's output with the target distribution of comparative probabilities, and improves state-of-the-art performance while maintaining high performance with an efficient subset of comparisons.

Abstract

Automated assessment in natural language generation is a challenging task. Instruction-tuned large language models (LLMs) have shown promise in reference-free evaluation, particularly through comparative assessment. However, the quadratic computational complexity of pairwise comparisons limits its scalability. To address this, efficient comparative assessment has been explored by applying comparative strategies on zero-shot LLM probabilities. We propose a framework for finetuning LLMs for comparative assessment to align the model's output with the target distribution of comparative probabilities. By training on soft probabilities, our approach improves state-of-the-art performance while maintaining high performance with an efficient subset of comparisons.

Finetuning LLMs for Comparative Assessment Tasks

TL;DR

This work proposes a framework for finetuning LLMs for comparative assessment to align the model's output with the target distribution of comparative probabilities, and improves state-of-the-art performance while maintaining high performance with an efficient subset of comparisons.

Abstract

Automated assessment in natural language generation is a challenging task. Instruction-tuned large language models (LLMs) have shown promise in reference-free evaluation, particularly through comparative assessment. However, the quadratic computational complexity of pairwise comparisons limits its scalability. To address this, efficient comparative assessment has been explored by applying comparative strategies on zero-shot LLM probabilities. We propose a framework for finetuning LLMs for comparative assessment to align the model's output with the target distribution of comparative probabilities. By training on soft probabilities, our approach improves state-of-the-art performance while maintaining high performance with an efficient subset of comparisons.
Paper Structure (19 sections, 4 equations, 5 figures, 6 tables)

This paper contains 19 sections, 4 equations, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. LLM comparative assessment
  4. Scoring methods
  5. Finetuning Systems
  6. Experiments
  7. Data
  8. Models
  9. Results
  10. Conclusions
  11. Limitations
  12. Ethics statement
  13. Impact of $\gamma$
  14. Relationship between PoE-BT and Thurstone-Mosteller
  15. Prompts
  16. ...and 4 more sections

Figures (5)

  • Figure 1: USMLE response time estimation: Efficient comparisons with Llama-3.1.
  • Figure 2: CMCQRD difficulty estimation: Efficient comparisons with Llama-3.1.
  • Figure 3: Impact of distribution of training probabilities based on choice of $\gamma$ in sigmoid.
  • Figure 4: Linear mapping between $\sigma$ and $\Phi$.
  • Figure 5: Relationship of scores (from zero-shot GPT-4o mini) using POE-BT and POE-TM for response time estimation.