Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Zero-Shot Strategies for Length-Controllable Summarization

Fabian Retkowski, Alexander Waibel

TL;DR

The paper addresses the challenge of zero-shot length controllability in abstractive summarization by evaluating LLMs across multiple length measures and proposing practical, non-fine-tuning methods. It introduces a formal framework of length measures, including structural and granular targets, and develops length-approximation, target-adjustment, sample-filtering, and automated-revision techniques, plus integrated recipe combinations. Empirical results with LLaMA-3-8B-Instruct on YTSeg and CNN/DM show that these methods substantially improve length adherence while preserving or improving summary quality, with structural measures being easiest to control and granular measures requiring the most care. The work demonstrates the practicality of zero-shot length control, offering scalable, API-friendly strategies for reliable, length-constrained summarization in real-world applications.

Abstract

Large language models (LLMs) struggle with precise length control, particularly in zero-shot settings. We conduct a comprehensive study evaluating LLMs' length control capabilities across multiple measures and propose practical methods to improve controllability. Our experiments with LLaMA 3 reveal stark differences in length adherence across measures and highlight inherent biases of the model. To address these challenges, we introduce a set of methods: length approximation, target adjustment, sample filtering, and automated revisions. By combining these methods, we demonstrate substantial improvements in length compliance while maintaining or enhancing summary quality, providing highly effective zero-shot strategies for precise length control without the need for model fine-tuning or architectural changes. With our work, we not only advance our understanding of LLM behavior in controlled text generation but also pave the way for more reliable and adaptable summarization systems in real-world applications.

Zero-Shot Strategies for Length-Controllable Summarization

TL;DR

The paper addresses the challenge of zero-shot length controllability in abstractive summarization by evaluating LLMs across multiple length measures and proposing practical, non-fine-tuning methods. It introduces a formal framework of length measures, including structural and granular targets, and develops length-approximation, target-adjustment, sample-filtering, and automated-revision techniques, plus integrated recipe combinations. Empirical results with LLaMA-3-8B-Instruct on YTSeg and CNN/DM show that these methods substantially improve length adherence while preserving or improving summary quality, with structural measures being easiest to control and granular measures requiring the most care. The work demonstrates the practicality of zero-shot length control, offering scalable, API-friendly strategies for reliable, length-constrained summarization in real-world applications.

Abstract

Large language models (LLMs) struggle with precise length control, particularly in zero-shot settings. We conduct a comprehensive study evaluating LLMs' length control capabilities across multiple measures and propose practical methods to improve controllability. Our experiments with LLaMA 3 reveal stark differences in length adherence across measures and highlight inherent biases of the model. To address these challenges, we introduce a set of methods: length approximation, target adjustment, sample filtering, and automated revisions. By combining these methods, we demonstrate substantial improvements in length compliance while maintaining or enhancing summary quality, providing highly effective zero-shot strategies for precise length control without the need for model fine-tuning or architectural changes. With our work, we not only advance our understanding of LLM behavior in controlled text generation but also pave the way for more reliable and adaptable summarization systems in real-world applications.
Paper Structure (79 sections, 15 equations, 8 figures, 26 tables)

This paper contains 79 sections, 15 equations, 8 figures, 26 tables.

Table of Contents

  1. Introduction
  2. Related Research
  3. Sample Filtering.
  4. Target Adjustment.
  5. Automated Revisions.
  6. Methodology
  7. Length Measures
  8. Length Approximation
  9. Character Length Approximation
  10. Token Length Approximation
  11. Target Adjustment
  12. Word Target Adjustment
  13. Sample Filtering
  14. Selection Process
  15. Automated Revisions
  16. ...and 64 more sections

Figures (8)

  • Figure 1: Overview of proposed methods for length controllability as part of a summarization inference pipeline
  • Figure 2: Analysis of length deviation across different parameters on YTSeg with data downsampled for visualization. Target summary lengths range from 25 to 200 words. Smoothed trend line using local regression (LOESS).
  • Figure 3:
  • Figure 4: Length Compliance with the Number of Revisions or Samples on YTSeg
  • Figure A1: Simple Prompting
  • ...and 3 more figures