GLIMMER: Incorporating Graph and Lexical Features in Unsupervised Multi-Document Summarization

TL;DR

GLIMMER presents an unsupervised, CPU-friendly approach to multi-document summarization that constructs a sentence graph, identifies semantic clusters via graph cuts using lexical features, and generates cluster-level summaries through a word-graph path optimization. It eliminates reliance on large pretraining data, achieves competitive or superior ROUGE and BERTScore performance against non-neural baselines and zero-shot models, and demonstrates strong readability and informativeness in human evaluations. The method scales across domains (news and scientific text) and offers substantial speed advantages over neural models, with code available for public use. The work also analyzes ablations, compares with ChatGPT, and discusses ethical considerations for deployment.

Abstract

Pre-trained language models are increasingly being used in multi-document summarization tasks. However, these models need large-scale corpora for pre-training and are domain-dependent. Other non-neural unsupervised summarization approaches mostly rely on key sentence extraction, which can lead to information loss. To address these challenges, we propose a lightweight yet effective unsupervised approach called GLIMMER: a Graph and LexIcal features based unsupervised Multi-docuMEnt summaRization approach. It first constructs a sentence graph from the source documents, then automatically identifies semantic clusters by mining low-level features from raw texts, thereby improving intra-cluster correlation and the fluency of generated sentences. Finally, it summarizes clusters into natural sentences. Experiments conducted on Multi-News, Multi-XScience and DUC-2004 demonstrate that our approach outperforms existing unsupervised approaches. Furthermore, it surpasses state-of-the-art pre-trained multi-document summarization models (e.g. PEGASUS and PRIMERA) under zero-shot settings in terms of ROUGE scores. Additionally, human evaluations indicate that summaries generated by GLIMMER achieve high readability and informativeness scores. Our code is available at https://github.com/Oswald1997/GLIMMER.

Figures (7)

• Figure 1: Illustration of GLIMMER. There are three basic steps: sentence graph construction, semantic cluster identification and cluster summarization.
• Figure 2: Comparisons between reference and generated summaries.
• Figure 3: Human evaluation guideline
• Figure 4: Distributions of human evaluation scores.
• Figure 5: Hallucinations in ChatGPT.