Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Papers-to-Posts: Supporting Detailed Long-Document Summarization with an Interactive LLM-Powered Source Outline

Marissa Radensky, Daniel S. Weld, Joseph Chee Chang, Pao Siangliulue, Jonathan Bragg

TL;DR

This work introduces interactive reverse source outlines as a mixed-initiative mechanism for grounded long-document summarization and implements it in Papers-to-Posts, a system for authoring research-paper blog posts. The approach maintains the source narrative through an LLM-generated reverse outline and allows iterative content selection and drafting in a plan-draft-revise cycle. Two user studies (lab with 20 participants and deployment with 26 participants across 37 blog posts) show that Papers-to-Posts increases editing power and, under time pressure, improves satisfaction with content coverage, while enabling easier incorporation of source content. The results highlight the value of grounding and controllable content selection for detailed summaries, while also revealing that full flexibility from baseline tools remains useful in some contexts; limitations include parsing accuracy and the absence of figures, guiding future enhancements toward hierarchical outlines and broader domain evaluation.

Abstract

Compressing long and technical documents (e.g., >10 pages) into shorter-form articles (e.g., <2 pages) is critical for communicating information to different audiences, for example, blog posts of scientific research paper or legal briefs of dense court proceedings. While large language models (LLMs) are powerful tools for condensing large amounts of text, current interfaces to these models lack support for understanding and controlling what content is included in a detailed summarizing article. Such capability is especially important for detail- and technical-oriented domains, in which tactical selection and coherent synthesis of key details is critical for effective communication to the target audience. For this, we present interactive reverse source outlines, a novel mechanism for controllable long-form summarization featuring outline bullet points with automatic point selections that the user can iteratively adjust to obtain an article with the desired content coverage. We implement this mechanism in Papers-to-Posts, a new LLM-powered system for authoring research-paper blog posts. Through a within-subjects lab study (n=20) and a between-subjects deployment study (n=37 blog posts, 26 participants), we compare Papers-to-Posts to a strong baseline tool that provides an LLM-generated draft and access to free-form prompting. Under time constraints, Papers-to-Posts significantly increases writer satisfaction with blog post quality, particularly with respect to content coverage. Furthermore, quantitative results showed an increase in editing power (change in text for an amount of time or writing actions) while using Papers-to-Posts, and qualitative results showed that participants found incorporating key research-paper insights in their blog posts easier while using Papers-to-Posts.

Papers-to-Posts: Supporting Detailed Long-Document Summarization with an Interactive LLM-Powered Source Outline

TL;DR

This work introduces interactive reverse source outlines as a mixed-initiative mechanism for grounded long-document summarization and implements it in Papers-to-Posts, a system for authoring research-paper blog posts. The approach maintains the source narrative through an LLM-generated reverse outline and allows iterative content selection and drafting in a plan-draft-revise cycle. Two user studies (lab with 20 participants and deployment with 26 participants across 37 blog posts) show that Papers-to-Posts increases editing power and, under time pressure, improves satisfaction with content coverage, while enabling easier incorporation of source content. The results highlight the value of grounding and controllable content selection for detailed summaries, while also revealing that full flexibility from baseline tools remains useful in some contexts; limitations include parsing accuracy and the absence of figures, guiding future enhancements toward hierarchical outlines and broader domain evaluation.

Abstract

Compressing long and technical documents (e.g., >10 pages) into shorter-form articles (e.g., <2 pages) is critical for communicating information to different audiences, for example, blog posts of scientific research paper or legal briefs of dense court proceedings. While large language models (LLMs) are powerful tools for condensing large amounts of text, current interfaces to these models lack support for understanding and controlling what content is included in a detailed summarizing article. Such capability is especially important for detail- and technical-oriented domains, in which tactical selection and coherent synthesis of key details is critical for effective communication to the target audience. For this, we present interactive reverse source outlines, a novel mechanism for controllable long-form summarization featuring outline bullet points with automatic point selections that the user can iteratively adjust to obtain an article with the desired content coverage. We implement this mechanism in Papers-to-Posts, a new LLM-powered system for authoring research-paper blog posts. Through a within-subjects lab study (n=20) and a between-subjects deployment study (n=37 blog posts, 26 participants), we compare Papers-to-Posts to a strong baseline tool that provides an LLM-generated draft and access to free-form prompting. Under time constraints, Papers-to-Posts significantly increases writer satisfaction with blog post quality, particularly with respect to content coverage. Furthermore, quantitative results showed an increase in editing power (change in text for an amount of time or writing actions) while using Papers-to-Posts, and qualitative results showed that participants found incorporating key research-paper insights in their blog posts easier while using Papers-to-Posts.
Paper Structure (97 sections, 28 figures)

This paper contains 97 sections, 28 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Human-LLM Text Summarization
  4. Outlines in Human-LLM Writing
  5. Human-LLM Scientific Writing
  6. Scientific Blog Posts
  7. The Papers-to-Posts System
  8. Design Goals
  9. Implementation Details
  10. Walkthrough of Warm Start Step
  11. Preprocessing
  12. Outline generation
  13. Outline point selection
  14. Summary generation
  15. Walkthrough of Planning Step
  16. ...and 82 more sections

Figures (28)

  • Figure 1: Interactive reverse source outlines in the Papers-to-Posts system. Users input a long-form source document (research paper) and the system provides (a) a WARM START for the summarization task by generating a reverse document outline. The system then produces a draft summary article (blog post) with sections based on outline bullet points (and associated source paragraphs) selected by the LLM. Given the draft, users can perform two main actions: (b) SUMMARY REVISION, where users adjust the system's bullet point selection for the summary, triggering system re-generation of the summary; and (c) SECTION ADDITION, where users provide a header for a new desired section, based on which the system selects bullet points and generates a draft of the section content. Users may also edit the draft manually.
  • Figure 2: Comparing interactive reverse source outlines in Papers-to-Posts (blue, upper right) with other affordances for control of LLM summarization, in terms of 1) how grounded the affordance is in the source narrative and 2) the length of the output summary that is supported. Papers-to-Posts is the first LLM-powered tool to provide writers with highly grounded control over a detailed summary.
  • Figure 3: Papers-to-Posts' user interface. a) The Blog-Post Area, where the user writes the blog post in sections. b) The Planning-Support Area, which contains both the interactive paper outline and original paper text. c1) The Drafting-Support Area, which contains inputs and outputs for generating text. c2) The Drafting-Support Area continued, seen if one scrolls below the c1 area. d) The Revising-Support Area, which contains inputs and outputs for modifying text and is located below the Drafting-Support Area. (Not Pictured: The modification output appears below the modification buttons, with a button to copy the output text and buttons to view previous modifications and their inputs.)
  • Figure 4: The baseline tool's user interface, consisting of areas for writing the blog post (left), viewing the paper (top right), and providing instructions to the LLM for generating writing (bottom right).
  • Figure 5: Survey responses to 7-point Likert-type questions regarding design goals in the a-d) lab study and e-h) deployment study. Responses are shown for both the treatment and baseline conditions.
  • ...and 23 more figures