Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PaperWeaver: Enriching Topical Paper Alerts by Contextualizing Recommended Papers with User-collected Papers

Yoonjoo Lee, Hyeonsu B. Kang, Matt Latzke, Juho Kim, Jonathan Bragg, Joseph Chee Chang, Pao Siangliulue

TL;DR

PaperWeaver addresses the lack of personalized context in standard paper alerts by generating contextualized descriptions anchored to user folders. It uses an LLM-based pipeline to infer interests from collected papers, extract folder-relevant problems, methods, and findings, and produce three types of paper-paper or aspect-based descriptions, including citance-based and pseudo-citance comparisons. In a within-subjects study with 15 CS researchers, PaperWeaver improved understanding of how recommended papers relate to a user's research context and supported richer connections between recommended and collected papers, albeit with some increased cognitive demand and occasional hallucinations that participants cross-verify. The work contributes a practical, extensible approach to sensemaking in literature reviews and suggests broad applicability beyond alerts to related-work reading and collaboration workflows.

Abstract

With the rapid growth of scholarly archives, researchers subscribe to "paper alert" systems that periodically provide them with recommendations of recently published papers that are similar to previously collected papers. However, researchers sometimes struggle to make sense of nuanced connections between recommended papers and their own research context, as existing systems only present paper titles and abstracts. To help researchers spot these connections, we present PaperWeaver, an enriched paper alerts system that provides contextualized text descriptions of recommended papers based on user-collected papers. PaperWeaver employs a computational method based on Large Language Models (LLMs) to infer users' research interests from their collected papers, extract context-specific aspects of papers, and compare recommended and collected papers on these aspects. Our user study (N=15) showed that participants using PaperWeaver were able to better understand the relevance of recommended papers and triage them more confidently when compared to a baseline that presented the related work sections from recommended papers.

PaperWeaver: Enriching Topical Paper Alerts by Contextualizing Recommended Papers with User-collected Papers

TL;DR

PaperWeaver addresses the lack of personalized context in standard paper alerts by generating contextualized descriptions anchored to user folders. It uses an LLM-based pipeline to infer interests from collected papers, extract folder-relevant problems, methods, and findings, and produce three types of paper-paper or aspect-based descriptions, including citance-based and pseudo-citance comparisons. In a within-subjects study with 15 CS researchers, PaperWeaver improved understanding of how recommended papers relate to a user's research context and supported richer connections between recommended and collected papers, albeit with some increased cognitive demand and occasional hallucinations that participants cross-verify. The work contributes a practical, extensible approach to sensemaking in literature reviews and suggests broad applicability beyond alerts to related-work reading and collaboration workflows.

Abstract

With the rapid growth of scholarly archives, researchers subscribe to "paper alert" systems that periodically provide them with recommendations of recently published papers that are similar to previously collected papers. However, researchers sometimes struggle to make sense of nuanced connections between recommended papers and their own research context, as existing systems only present paper titles and abstracts. To help researchers spot these connections, we present PaperWeaver, an enriched paper alerts system that provides contextualized text descriptions of recommended papers based on user-collected papers. PaperWeaver employs a computational method based on Large Language Models (LLMs) to infer users' research interests from their collected papers, extract context-specific aspects of papers, and compare recommended and collected papers on these aspects. Our user study (N=15) showed that participants using PaperWeaver were able to better understand the relevance of recommended papers and triage them more confidently when compared to a baseline that presented the related work sections from recommended papers.
Paper Structure (55 sections, 6 figures)

This paper contains 55 sections, 6 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Facilitating Broad Scholarly Exploration and Paper Discovery
  4. Reusing Related Work Sections for Deeper Scholarly Sensemaking
  5. Providing contextualized explanation
  6. Formative Study
  7. Procedure and Apparatus
  8. Formative Study Findings
  9. Challenges in Making Sense of Existing Paper Alerts
  10. Surfacing Relevant Aspects when Summarizing Recommended Papers
  11. Comparing and Contrasting Recommended and User-collected Papers
  12. Bringing New Perspectives to Previously Collected Papers
  13. Design Goals
  14. PaperWeaver
  15. Example User Scenario
  16. ...and 40 more sections

Figures (6)

  • Figure 1: In contrast to existing topical paper alert systems that show descriptions with no personalized context for the recommended papers, PaperWeaver provides contextualized descriptions that surface the relevance of recommended papers and anchor them to familiar user-collected folders to help users better make sense of recommended papers.
  • Figure 2: PaperWeaver's paper alert interface. The page shows the folder title and description (A) and a list of recommended papers. Each recommended paper is shown on a paper card with the title, authors, venue, and publication year. The bottom of the card features descriptions in three tabs: Relate to Paper (B) allows users to select a paper from existing papers in the folder with the description related to the recommended paper. These descriptions are paper-paper Descriptions Based on Citances and Paper-paper Descriptions via Generated Pseudo-citances that focus on specific relationships between two papers. The description assigns the label "Paper A" to the recommended paper and "Paper B" to the existing paper in the library and highlights the references in different colors; Problem, method, and findings (C) describes the recommended papers in three aspects--- problem, method, and findings---related to the folder context with contextualized Aspect-based Paper Summaries; Abstract (D) shows the unmodified abstract of the paper. The user can save a paper to the library (E) and take notes about the recommended paper (F).
  • Figure 3: Overview of PaperWeaver's pipeline to generate contextualized descriptions. If there is a citation from the recommended paper to a collected paper, PaperWeaver generates paper-paper descriptions based on citances using citing paragraph. If there are no citances, PaperWeaver synthesizes pseudo-citing sentences that shows the relationship between recommended paper and relevant collected paper to make paper-paper descriptions via generated pseudo-citances. If there are no relevant collected papers, PaperWeaver generates contextualized aspect-based paper summaries with the folder's overall context. Prompt T1-T6 are in the Appendix \ref{['appendix:prompts']}.
  • Figure 4: In the baseline condition, participants also see the recommended papers in a similar paper card to PaperWeaver paper alert interface. Instead of the three tabs with different types of explanation, there are two tabs: Abstract that shows the abstract of the recommended paper and Related work that shows the text of the related work section of the recommended paper. The rest of the functionalities remain the same across conditions.
  • Figure 5: Results of the post-survey showed that participants found various benefits when using PaperWeaver compared to a strong baseline. **, *, and ns indicate significance of $p < 0.01$, $p < 0.05$, and $p > 0.05$, respectively.
  • ...and 1 more figures