Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Teaching Large Language Models Number-Focused Headline Generation With Key Element Rationales

Zhen Qian, Xiuzhen Zhang, Xiaofei Xu, Feng Xia

TL;DR

This paper tackles number-focused headline generation, where both textual quality and numerical accuracy are critical. It introduces TEN, a chain-of-thought framework that decomposes headlines into Topic, Entities, and Numerical reasoning, and uses a teacher-student distillation pipeline to automatically produce TEN rationales and headlines. A Direct Preference Optimization stage refines rationale quality to improve topic alignment and numerical correctness. Experiments on NumHG and XSum show that TEN substantially improves numerical accuracy while maintaining strong text quality, demonstrating the practical viability of rationale-augmented, numerically grounded headline generation. The work advances how LLMs can jointly reason about content and numbers in abstractive summarization, with implications for more reliable news generation and evaluation.

Abstract

Number-focused headline generation is a summarization task requiring both high textual quality and precise numerical accuracy, which poses a unique challenge for Large Language Models (LLMs). Existing studies in the literature focus only on either textual quality or numerical reasoning and thus are inadequate to address this challenge. In this paper, we propose a novel chain-of-thought framework for using rationales comprising key elements of the Topic, Entities, and Numerical reasoning (TEN) in news articles to enhance the capability for LLMs to generate topic-aligned high-quality texts with precise numerical accuracy. Specifically, a teacher LLM is employed to generate TEN rationales as supervision data, which are then used to teach and fine-tune a student LLM. Our approach teaches the student LLM automatic generation of rationales with enhanced capability for numerical reasoning and topic-aligned numerical headline generation. Experiments show that our approach achieves superior performance in both textual quality and numerical accuracy.

Teaching Large Language Models Number-Focused Headline Generation With Key Element Rationales

TL;DR

This paper tackles number-focused headline generation, where both textual quality and numerical accuracy are critical. It introduces TEN, a chain-of-thought framework that decomposes headlines into Topic, Entities, and Numerical reasoning, and uses a teacher-student distillation pipeline to automatically produce TEN rationales and headlines. A Direct Preference Optimization stage refines rationale quality to improve topic alignment and numerical correctness. Experiments on NumHG and XSum show that TEN substantially improves numerical accuracy while maintaining strong text quality, demonstrating the practical viability of rationale-augmented, numerically grounded headline generation. The work advances how LLMs can jointly reason about content and numbers in abstractive summarization, with implications for more reliable news generation and evaluation.

Abstract

Number-focused headline generation is a summarization task requiring both high textual quality and precise numerical accuracy, which poses a unique challenge for Large Language Models (LLMs). Existing studies in the literature focus only on either textual quality or numerical reasoning and thus are inadequate to address this challenge. In this paper, we propose a novel chain-of-thought framework for using rationales comprising key elements of the Topic, Entities, and Numerical reasoning (TEN) in news articles to enhance the capability for LLMs to generate topic-aligned high-quality texts with precise numerical accuracy. Specifically, a teacher LLM is employed to generate TEN rationales as supervision data, which are then used to teach and fine-tune a student LLM. Our approach teaches the student LLM automatic generation of rationales with enhanced capability for numerical reasoning and topic-aligned numerical headline generation. Experiments show that our approach achieves superior performance in both textual quality and numerical accuracy.

Paper Structure

This paper contains 19 sections, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Headline Generation
  4. CoT Prompting for Rationale Generation
  5. Learning from Teacher LLM Generated Rationales
  6. Methodology
  7. Teacher LLM generation of TEN rationales
  8. Fine-Tuning Student LLMs
  9. Experiments
  10. Datasets and Evaluation Metrics
  11. Baselines
  12. Experiment Results
  13. Performance of TEN rationales and Teacher-student knowledge distillation
  14. Ablation study
  15. Error analysis and case study
  16. ...and 4 more sections

Figures (5)

  • Figure 1: An example TEN Rationale for key elements of Topic (green), Entities (blue) and Numerical reasoning (purple).
  • Figure 2: Our TEN approach for automatic generation of rationales to enhance numerical headline generation.
  • Figure 3: The process for Teacher LLM to generate TEN rationales for fine-tuning student LLMs
  • Figure 4: TEN vs. NCL (Baseline) for rationale and headline generation
  • Figure 5: Flowchart for constructing preference data