Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Just What You Desire: Constrained Timeline Summarization with Self-Reflection for Enhanced Relevance

Muhammad Reza Qorib, Qisheng Hu, Hwee Tou Ng

TL;DR

Constrained timeline summarization (CTLS) addresses the problem of producing timeline outputs tailored to reader-specific constraints. The authors introduce CREST, a dataset with 235 timelines across 47 entities and 5 constraints per entity, and propose REACTS, a no-training LLM-based pipeline that summarizes each article per constraint, clusters constraint-related summaries into events, and selects the top $l$ clusters with $k$ sentences per cluster. A novel self-reflection step verifies adherence to the constraint, discarding non-adherent outputs, which significantly improves alignment-based ROUGE $F_1$ and date $F_1$ scores across models. The results show that REACTS outperforms a baseline that concatenates all articles and that self-reflection yields robust gains, suggesting practical applicability for real-time, constraint-driven timeline generation.

Abstract

Given news articles about an entity, such as a public figure or organization, timeline summarization (TLS) involves generating a timeline that summarizes the key events about the entity. However, the TLS task is too underspecified, since what is of interest to each reader may vary, and hence there is not a single ideal or optimal timeline. In this paper, we introduce a novel task, called Constrained Timeline Summarization (CTLS), where a timeline is generated in which all events in the timeline meet some constraint. An example of a constrained timeline concerns the legal battles of Tiger Woods, where only events related to his legal problems are selected to appear in the timeline. We collected a new human-verified dataset of constrained timelines involving 47 entities and 5 constraints per entity. We propose an approach that employs a large language model (LLM) to summarize news articles according to a specified constraint and cluster them to identify key events to include in a constrained timeline. In addition, we propose a novel self-reflection method during summary generation, demonstrating that this approach successfully leads to improved performance.

Just What You Desire: Constrained Timeline Summarization with Self-Reflection for Enhanced Relevance

TL;DR

Constrained timeline summarization (CTLS) addresses the problem of producing timeline outputs tailored to reader-specific constraints. The authors introduce CREST, a dataset with 235 timelines across 47 entities and 5 constraints per entity, and propose REACTS, a no-training LLM-based pipeline that summarizes each article per constraint, clusters constraint-related summaries into events, and selects the top clusters with sentences per cluster. A novel self-reflection step verifies adherence to the constraint, discarding non-adherent outputs, which significantly improves alignment-based ROUGE and date scores across models. The results show that REACTS outperforms a baseline that concatenates all articles and that self-reflection yields robust gains, suggesting practical applicability for real-time, constraint-driven timeline generation.

Abstract

Given news articles about an entity, such as a public figure or organization, timeline summarization (TLS) involves generating a timeline that summarizes the key events about the entity. However, the TLS task is too underspecified, since what is of interest to each reader may vary, and hence there is not a single ideal or optimal timeline. In this paper, we introduce a novel task, called Constrained Timeline Summarization (CTLS), where a timeline is generated in which all events in the timeline meet some constraint. An example of a constrained timeline concerns the legal battles of Tiger Woods, where only events related to his legal problems are selected to appear in the timeline. We collected a new human-verified dataset of constrained timelines involving 47 entities and 5 constraints per entity. We propose an approach that employs a large language model (LLM) to summarize news articles according to a specified constraint and cluster them to identify key events to include in a constrained timeline. In addition, we propose a novel self-reflection method during summary generation, demonstrating that this approach successfully leads to improved performance.

Paper Structure

This paper contains 38 sections, 3 figures, 13 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Timeline Summarization
  4. Direct Summarization
  5. Date-wise Approach
  6. Event Detection
  7. Query-Based Summarization
  8. Update Summarization
  9. Dataset
  10. Constraint Generation
  11. Event Annotation
  12. Event Filtering
  13. Problem Definition
  14. Method
  15. Event Summarization
  16. ...and 23 more sections

Figures (3)

  • Figure 1: The dataset creation process consists of three steps: constraint generation, event annotation, and event filtering. Human annotators are tasked with determining whether an event in a timeline adheres to a constraint or not. The list of events that adheres to a constraint becomes a constrained timeline.
  • Figure 2: Illustration of our method for constrained timeline summarization (REACTS). The method consists of four steps: event summarization, self-reflection, event clustering, and cluster and sentence selection.
  • Figure 3: Example of the annotation interface for the event-constraint pairs in Tiger Woods's timeline.