Toward Living Narrative Reviews: An Empirical Study of the Processes and Challenges in Updating Survey Articles in Computing Research

TL;DR

The paper tackles the problem of information staleness in computing survey articles by exploring how narrative reviews are authored and kept current. Through 11 retrospective interviews with survey authors and thematic analysis, it identifies four core authoring activities (search, appraisal, synthesis, interpretation) and three types of updates (empirical, structural, interpretative). It also evaluates potential roles for AI, finding that AI can automate routine tasks and assist as a surrogate or second opinion, but cannot yet replace expert, nuanced interpretation. The work offers a vision for living narrative reviews and outlines design considerations for AI-assisted updating to reduce effort while preserving expert judgment, with implications for tool developers, researchers, and publishers.

Abstract

Surveying prior literature to establish a foundation for new knowledge is essential for scholarly progress. However, survey articles are resource-intensive and challenging to create, and can quickly become outdated as new research is published, risking information staleness and inaccuracy. Keeping survey articles current with the latest evidence is therefore desirable, though there is a limited understanding of why, when, and how these surveys should be updated. Toward this end, through a series of in-depth retrospective interviews with 11 researchers, we present an empirical examination of the work practices in authoring and updating survey articles in computing research. We find that while computing researchers acknowledge the value in maintaining an updated survey, continuous updating remains unmanageable and misaligned with academic incentives. Our findings suggest key leverage points within current workflows that present opportunities for enabling technologies to facilitate more efficient and effective updates.