Analysis of Multidomain Abstractive Summarization Using Salience Allocation
Tohida Rehman, Raghubir Bose, Soumik Dey, Samiran Chattopadhyay
TL;DR
The paper investigates enhancing abstractive summarization by introducing SEASON, a salience-guided approach that allocates sentence salience and employs salience-aware cross-attention to steer generation. It benchmarks SEASON against strong pre-trained models (BART, PEGASUS, ProphetNet) on CNN/DailyMail, SAMSum, and Financial EDT datasets, using ROUGE, METEOR, BERTScore, and MoverScore to assess performance. Results indicate SEASON is highly competitive, delivering strong results on SAMSum and EDT and offering qualitative improvements in conciseness and factuality, with PEGASUS sometimes excelling on CNN/DailyMail. The work highlights the potential of salience allocation to improve cross-domain abstractive summarization and outlines future directions for scaling and applying the approach to broader text-generation tasks such as QA and chatbots.
Abstract
This paper explores the realm of abstractive text summarization through the lens of the SEASON (Salience Allocation as Guidance for Abstractive SummarizatiON) technique, a model designed to enhance summarization by leveraging salience allocation techniques. The study evaluates SEASON's efficacy by comparing it with prominent models like BART, PEGASUS, and ProphetNet, all fine-tuned for various text summarization tasks. The assessment is conducted using diverse datasets including CNN/Dailymail, SAMSum, and Financial-news based Event-Driven Trading (EDT), with a specific focus on a financial dataset containing a substantial volume of news articles from 2020/03/01 to 2021/05/06. This paper employs various evaluation metrics such as ROUGE, METEOR, BERTScore, and MoverScore to evaluate the performance of these models fine-tuned for generating abstractive summaries. The analysis of these metrics offers a thorough insight into the strengths and weaknesses demonstrated by each model in summarizing news dataset, dialogue dataset and financial text dataset. The results presented in this paper not only contribute to the evaluation of the SEASON model's effectiveness but also illuminate the intricacies of salience allocation techniques across various types of datasets.