Explanatory Summarization with Discourse-Driven Planning

TL;DR

This work tackles the challenge of generating lay summaries with controlled explanatory content for scientific documents. It introduces a discourse-driven planning framework that uses RST and the Question Under Discussion to create EDU-level plans guiding explanations, implemented in Plan-Output and Plan-Input variants. Across SciNews, eLife, and PLOS, the approach improves summary quality, readability, and factual consistency while reducing hallucinations, with Plan-Input often performing best. The method emphasizes controllability of explanatory content through plan design and incorporates external knowledge verification, underscoring its practical impact for safer, more accessible science communication.

Abstract

Lay summaries for scientific documents typically include explanations to help readers grasp sophisticated concepts or arguments. However, current automatic summarization methods do not explicitly model explanations, which makes it difficult to align the proportion of explanatory content with human-written summaries. In this paper, we present a plan-based approach that leverages discourse frameworks to organize summary generation and guide explanatory sentences by prompting responses to the plan. Specifically, we propose two discourse-driven planning strategies, where the plan is conditioned as part of the input or part of the output prefix, respectively. Empirical experiments on three lay summarization datasets show that our approach outperforms existing state-of-the-art methods in terms of summary quality, and it enhances model robustness, controllability, and mitigates hallucination.

Figures (17)

• Figure 1: An excerpt of a source document paired with its summary (bottom); the explanatory sentence and its target are highlighted in green and orange, respectively. The RST tree (top) for the text corresponding to the two sentences shows they are linked by the rhetorical relation Elaboration.
• Figure 2: We use DMRSTliu-etal-2021-dmrst to extract explanatory ($e$) EDUs and their target ($t$) EDUs from reference summaries, and then feed this data into GPT-4o to generate plans ($b$).
• Figure 3: Summary quality as a function of different RST parsers.
• Figure 4: Summary quality as a function of different question generation methods.
• Figure 5: Human evaluation results along different dimensions of summary quality.