Sentence Embeddings as an intermediate target in end-to-end summarisation
Maciej Zembrzuski, Saad Mahamood
TL;DR
This paper tackles content-selection challenges in end-to-end summarisation of large, unstructured user hotel reviews. It introduces USEsum, a two-stage pipeline that first uses an extractive module based on Universal Sentence Encoder embeddings and angle-based information gain to select three salient sentences, then feeds them to a Transformer-based abstractive generator to produce a one-sentence Unique Selling Point description. The approach is evaluated on the USEG dataset, showing competitive or superior performance to existing extractive and end-to-end methods across several automatic metrics, with human evaluation highlighting strong semantic alignment. The work demonstrates the utility of sentence-embedding-based representations for content selection in low-overlap, highly compressed multi-source reviews, and suggests promising directions for improving end-to-end summarisation in large inputs and across languages.
Abstract
Current neural network-based methods to the problem of document summarisation struggle when applied to datasets containing large inputs. In this paper we propose a new approach to the challenge of content-selection when dealing with end-to-end summarisation of user reviews of accommodations. We show that by combining an extractive approach with externally pre-trained sentence level embeddings in an addition to an abstractive summarisation model we can outperform existing methods when this is applied to the task of summarising a large input dataset. We also prove that predicting sentence level embedding of a summary increases the quality of an end-to-end system for loosely aligned source to target corpora, than compared to commonly predicting probability distributions of sentence selection.