Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FactAlign: Long-form Factuality Alignment of Large Language Models

Chao-Wei Huang, Yun-Nung Chen

TL;DR

This paper proposes FactAlign, a novel alignment framework designed to enhance the factuality of LLMs' long-form responses while maintaining their helpfulness, and introduces fKTO, a fine-grained, sentence-level alignment algorithm that extends the Kahneman-Tversky Optimization (KTO) alignment method.

Abstract

Large language models have demonstrated significant potential as the next-generation information access engines. However, their reliability is hindered by issues of hallucination and generating non-factual content. This is particularly problematic in long-form responses, where assessing and ensuring factual accuracy is complex. In this paper, we address this gap by proposing FactAlign, a novel alignment framework designed to enhance the factuality of LLMs' long-form responses while maintaining their helpfulness. We introduce fKTO, a fine-grained, sentence-level alignment algorithm that extends the Kahneman-Tversky Optimization (KTO) alignment method. Leveraging recent advances in automatic factuality evaluation, FactAlign utilizes fine-grained factuality assessments to guide the alignment process. Our experiments on open-domain prompts and information-seeking questions demonstrate that FactAlign significantly improves the factual accuracy of LLM responses while also improving their helpfulness. Further analyses identify that FactAlign is capable of training LLMs to provide more information without losing factual precision, thus improving the factual F1 score. Our source code, datasets, and trained models are publicly available at https://github.com/MiuLab/FactAlign

FactAlign: Long-form Factuality Alignment of Large Language Models

TL;DR

This paper proposes FactAlign, a novel alignment framework designed to enhance the factuality of LLMs' long-form responses while maintaining their helpfulness, and introduces fKTO, a fine-grained, sentence-level alignment algorithm that extends the Kahneman-Tversky Optimization (KTO) alignment method.

Abstract

Large language models have demonstrated significant potential as the next-generation information access engines. However, their reliability is hindered by issues of hallucination and generating non-factual content. This is particularly problematic in long-form responses, where assessing and ensuring factual accuracy is complex. In this paper, we address this gap by proposing FactAlign, a novel alignment framework designed to enhance the factuality of LLMs' long-form responses while maintaining their helpfulness. We introduce fKTO, a fine-grained, sentence-level alignment algorithm that extends the Kahneman-Tversky Optimization (KTO) alignment method. Leveraging recent advances in automatic factuality evaluation, FactAlign utilizes fine-grained factuality assessments to guide the alignment process. Our experiments on open-domain prompts and information-seeking questions demonstrate that FactAlign significantly improves the factual accuracy of LLM responses while also improving their helpfulness. Further analyses identify that FactAlign is capable of training LLMs to provide more information without losing factual precision, thus improving the factual F1 score. Our source code, datasets, and trained models are publicly available at https://github.com/MiuLab/FactAlign
Paper Structure (36 sections, 8 equations, 3 figures, 5 tables)

This paper contains 36 sections, 8 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Language Model Alignment
  4. Factuality of Langage Models
  5. Preliminaries
  6. Long-form Factuality
  7. Kahneman-Tversky Optimization
  8. FactAlign: Aligning Language Models for Long-form Factuality
  9. Automatic Long-form Factuality Evaluator
  10. Atomic Statement Decomposition
  11. Query Generation
  12. Relevant Knowledge Search
  13. Final Factuality Assessment
  14. Long-form Factuality Alignment
  15. Response-level Alignment
  16. ...and 21 more sections

Figures (3)

  • Figure 1: An example of the evaluation long-form factuality. The long-form response is broken down into subclaims and verified separately. The factual precision score can be calculated as the precision of all subclaims.
  • Figure 2: An overview of our FactAlign framework. Top: the pipeline for long-form facutality assessment. Bottom: the long-form factuality alignment process.
  • Figure 3: The precision-recall curve with varying ratios of data mixture. SFT denotes the supervised fine-tuned baseline. The labels denote the ratio of the precision data points used.