Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Concise and Sufficient Sub-Sentence Citations for Retrieval-Augmented Generation

Guo Chen, Qiuyuan Li, Qiuxian Li, Hongliang Dai, Xiang Chen, Piji Li

TL;DR

The paper tackles verification challenges in retrieval-augmented generation by introducing sub-sentence, concise yet sufficient citations to replace broader sentence- or paragraph-level citations. It develops annotation principles and a manually labeled dataset, then implements a credit-model–driven, data-augmentation pipeline that leverages open-source LLMs to generate high-quality fine-grained citations and trains a citation model via LoRA fine-tuning. Empirical results show improved lexical and semantic alignment and readability, with scalability gains as the training data increases, demonstrating strong cross-model generalization. This work enhances verifiability in RAG systems and lays groundwork for future multimodal and multilingual attribution research.

Abstract

In retrieval-augmented generation (RAG) question answering systems, generating citations for large language model (LLM) outputs enhances verifiability and helps users identify potential hallucinations. However, we observe two problems in the citations produced by existing attribution methods. First, the citations are typically provided at the sentence or even paragraph level. Long sentences or paragraphs may include a substantial amount of irrelevant content. Second, sentence-level citations may omit information that is essential for verifying the output, forcing users to read the surrounding context. In this paper, we propose generating sub-sentence citations that are both concise and sufficient, thereby reducing the effort required by users to confirm the correctness of the generated output. To this end, we first develop annotation guidelines for such citations and construct a corresponding dataset. Then, we propose an attribution framework for generating citations that adhere to our standards. This framework leverages LLMs to automatically generate fine-tuning data for our task and employs a credit model to filter out low-quality examples. Our experiments on the constructed dataset demonstrate that the propose approach can generate high-quality and more readable citations.

Concise and Sufficient Sub-Sentence Citations for Retrieval-Augmented Generation

TL;DR

The paper tackles verification challenges in retrieval-augmented generation by introducing sub-sentence, concise yet sufficient citations to replace broader sentence- or paragraph-level citations. It develops annotation principles and a manually labeled dataset, then implements a credit-model–driven, data-augmentation pipeline that leverages open-source LLMs to generate high-quality fine-grained citations and trains a citation model via LoRA fine-tuning. Empirical results show improved lexical and semantic alignment and readability, with scalability gains as the training data increases, demonstrating strong cross-model generalization. This work enhances verifiability in RAG systems and lays groundwork for future multimodal and multilingual attribution research.

Abstract

In retrieval-augmented generation (RAG) question answering systems, generating citations for large language model (LLM) outputs enhances verifiability and helps users identify potential hallucinations. However, we observe two problems in the citations produced by existing attribution methods. First, the citations are typically provided at the sentence or even paragraph level. Long sentences or paragraphs may include a substantial amount of irrelevant content. Second, sentence-level citations may omit information that is essential for verifying the output, forcing users to read the surrounding context. In this paper, we propose generating sub-sentence citations that are both concise and sufficient, thereby reducing the effort required by users to confirm the correctness of the generated output. To this end, we first develop annotation guidelines for such citations and construct a corresponding dataset. Then, we propose an attribution framework for generating citations that adhere to our standards. This framework leverages LLMs to automatically generate fine-tuning data for our task and employs a credit model to filter out low-quality examples. Our experiments on the constructed dataset demonstrate that the propose approach can generate high-quality and more readable citations.

Paper Structure

This paper contains 10 sections, 3 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data Annotation Principles
  4. Methodology
  5. Experiments
  6. Evaluation Metrics
  7. Fine-tuned Model Performance Validation
  8. Case Study on Fine-grained Attribution
  9. Ablation Study on Data Scalability
  10. Conclusion

Figures (3)

  • Figure 1: Comparison of cited content produced by an existing approach and our sub-sentence citation standard. The cited content is highlighted in red. Some unimportant content are omitted with "...".
  • Figure 2: Sub-sentence-level citations adhering to our annotation principles.
  • Figure 3: Overall procedure of our approach.