TLDR: Extreme Summarization of Scientific Documents

TL;DR

This paper defines TLDR generation as extreme summarization for scientific papers and introduces SciTLDR, a multi-target dataset containing author-written and expert-derived TLDRs across CS papers. It presents Catts, a title-scaffolding learning strategy that uses paper titles as auxiliary targets to improve TLDR generation under data-scarce conditions. Through extensive experiments with extractive and abstractive baselines, Catts shows improvements in automated Rouge metrics and human evaluators find Catts-generated TLDRs more informative and faithful than strong baselines, approaching the quality of expert summaries. The work highlights the value of multiple gold TLDR targets, analyzes information content via nugget categories, and demonstrates the potential of using extended input contexts and title scaffolding to advance extreme summarization in scientific domains.

Abstract

We introduce TLDR generation, a new form of extreme summarization, for scientific papers. TLDR generation involves high source compression and requires expert background knowledge and understanding of complex domain-specific language. To facilitate study on this task, we introduce SciTLDR, a new multi-target dataset of 5.4K TLDRs over 3.2K papers. SciTLDR contains both author-written and expert-derived TLDRs, where the latter are collected using a novel annotation protocol that produces high-quality summaries while minimizing annotation burden. We propose CATTS, a simple yet effective learning strategy for generating TLDRs that exploits titles as an auxiliary training signal. CATTS improves upon strong baselines under both automated metrics and human evaluations. Data and code are publicly available at https://github.com/allenai/scitldr.

Figures (4)

• Figure 1: An example tldr of a scientific paper. A tldr is typically composed of salient information (indicated by colored spans) found in the abstract, intro, and conclusion sections of a paper.
• Figure 2: Example of a reviewer comment rewritten as a tldr (best viewed in color). A peer review comment often begins with a summary of the paper which annotators use to compose a tldr. Annotators are trained to preserve the original reviewer's wording when possible (indicated by colored spans), and to avoid using any foogray!10 excess details or foored!20 criticism.
• Figure 3: Two example tldr-Auth and tldr-PR pairs with colored spans corresponding to nuggets in Table \ref{['fig:nugget-examples']} -- foored!20 A, foogreen!20 P, fooblue!10 C, fooyellow!40 D. On top, we see tldrs can have substantial lexical variation despite covering similar information content. On bottom, we naturally see even more variation when the information content differs.
• Figure 4: Training regimen for Catts.