InforME: Improving Informativeness of Abstractive Text Summarization With Informative Attention Guided by Named Entity Salience

TL;DR

This paper tackles the informativeness gap in abstractive text summarization by introducing two regularizers: an optimal transport–based informative attention that acts as reverse cross-attention to emphasize focal information from reference summaries, and an accumulative joint entropy reduction (AJER) that leverages named-entity salience to reduce uncertainty in entity sequences. Integrated into an end-to-end ATS framework with a BART-large backbone, the approach yields superior ROUGE performance on CNN/Daily Mail and competitive results on XSum, complemented by human evaluations that confirm increased informativeness and nuanced factuality analyses. The results suggest that guiding attention with informativeness from reference summaries and stabilizing named-entity representations can enhance the usefulness of generated summaries in practice, albeit with domain-specific trade-offs and extrinsic factual considerations. Overall, InforME offers a principled method for making abstractive summaries more information-rich by aligning model focus with salient content and entity structure.

Abstract

Abstractive text summarization is integral to the Big Data era, which demands advanced methods to turn voluminous and often long text data into concise but coherent and informative summaries for efficient human consumption. Despite significant progress, there is still room for improvement in various aspects. One such aspect is to improve informativeness. Hence, this paper proposes a novel learning approach consisting of two methods: an optimal transport-based informative attention method to improve learning focal information in reference summaries and an accumulative joint entropy reduction method on named entities to enhance informative salience. Experiment results show that our approach achieves better ROUGE scores compared to prior work on CNN/Daily Mail while having competitive results on XSum. Human evaluation of informativeness also demonstrates the better performance of our approach over a strong baseline. Further analysis gives insight into the plausible reasons underlying the evaluation results.

Figures (2)

• Figure 1: Illustration of an encoder-decoder with our methods, including optimal transport-based informative attention (carmine block) and accumulative joint entropy reduction (tealish block).
• Figure 2: Main user interface of informativeness evaluation.