XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages

TL;DR

XL-Sum addresses the scarcity of multilingual abstractive summarization data by assembling a 1-million pair BBC-based dataset spanning 44 languages. It uses a careful data-collection and extraction pipeline that leverages consistently styled summaries (bold introductory paragraphs) and rigorous heuristics to ensure input–summary alignment. The authors benchmark multilingual and low-resource training using mT5, reporting competitive ROUGE-2 results across languages and demonstrating transfer benefits between related languages. They also provide intrinsic quality analyses (abstractiveness, compression, redundancy) and human evaluation to validate data quality. The resource—dataset, curation tool, and model checkpoints—promises to advance multilingual summarization research, especially for low- and mid-resource languages.

Abstract

Contemporary works on abstractive text summarization have focused primarily on high-resource languages like English, mostly due to the limited availability of datasets for low/mid-resource ones. In this work, we present XL-Sum, a comprehensive and diverse dataset comprising 1 million professionally annotated article-summary pairs from BBC, extracted using a set of carefully designed heuristics. The dataset covers 44 languages ranging from low to high-resource, for many of which no public dataset is currently available. XL-Sum is highly abstractive, concise, and of high quality, as indicated by human and intrinsic evaluation. We fine-tune mT5, a state-of-the-art pretrained multilingual model, with XL-Sum and experiment on multilingual and low-resource summarization tasks. XL-Sum induces competitive results compared to the ones obtained using similar monolingual datasets: we show higher than 11 ROUGE-2 scores on 10 languages we benchmark on, with some of them exceeding 15, as obtained by multilingual training. Additionally, training on low-resource languages individually also provides competitive performance. To the best of our knowledge, XL-Sum is the largest abstractive summarization dataset in terms of the number of samples collected from a single source and the number of languages covered. We are releasing our dataset and models to encourage future research on multilingual abstractive summarization. The resources can be found at \url{https://github.com/csebuetnlp/xl-sum}.