Dont Add, dont Miss: Effective Content Preserving Generation from Pre-Selected Text Spans

TL;DR

This work advances Controlled Text Reduction (CTR) by addressing two bottlenecks: insufficient highlight preservation and noisy silver training data. It introduces three core components—highlights-oriented reinforcement learning (via an adapted Quark framework), highlights-sensitive decoding (a faithfulness-like lookahead biased toward pre-selected highlights), and GPT-4-based distillation to improve training data quality. Each component independently improves the model's adherence to and coverage of highlights, with GPT-4 distillation yielding the largest data-quality gains and enabling state-of-the-art CTR performance by over 30 ROUGE-L points while preserving coherence. The findings support deploying CTR as a modular building block in downstream tasks, enabling reliable, content-preserving generation conditioned on pre-selected input spans.

Abstract

The recently introduced Controlled Text Reduction (CTR) task isolates the text generation step within typical summarization-style tasks. It does so by challenging models to generate coherent text conforming to pre-selected content within the input text (``highlights''). This framing enables increased modularity in summarization-like tasks, allowing to couple a single CTR model with various content-selection setups and modules. However, there are currently no reliable CTR models, while the performance of the existing baseline for the task is mediocre, falling short of practical utility. Here, we address this gap by introducing a high-quality, open-source CTR model that tackles two prior key limitations: inadequate enforcement of the content-preservation constraint, and suboptimal silver training data. Addressing these, we amplify the content-preservation constraint in both training, via RL, and inference, via a controlled decoding strategy. Further, we substantially improve the silver training data quality via GPT-4 distillation. Overall, pairing the distilled dataset with the highlight-adherence strategies yields marked gains over the current baseline, of up to 30 ROUGE-L points, providing a reliable CTR model for downstream use.

Figures (7)

• Figure 1: Overview of our contributions, encompassing three modeling phases. Components introduced in our approach are denoted in blue. (a) We generate new target summaries using GPT-4, conditioned on the silver highlights in the original dataset. (b) During training, we fine-tune our model taking an RL approach, based on Quark lu2022quark. (c) During inference, we employ a highlights-centric controlled decoding algorithm.
• Figure 2: Demonstration of the Controlled Text Reduction task. The input consists of a source document and highlights (left), and the desirable output covers exclusively the highlighted content while preserving coherence (right). Borrowed and adapted from slobodkin-etal-2022-controlled.
• Figure 3: ROUGE, METEOR, and BertScore results on 50 instances from the CTR development set of GPT-4 models, for varying numbers of in-context examples in the prompt.
• Figure 4: The scores of fine-tuned Flan-T5H with controlled decoding compared to regular decoding at various beam sizes. We present a comparison of the generated summary to the highlights concatenation.
• Figure 5: Example of the data collection interface used by the crowd-workers to evaluate the fluency of summaries.