SWARM-SLR -- Streamlined Workflow Automation for Machine-actionable Systematic Literature Reviews

TL;DR

The Streamlined Workflow Automation for Machine-actionable Systematic Literature Reviews (SWARM-SLR) is proposed to crowdsource the improvement of SLR efficiency while maintaining scientific integrity in a state-of-the-art knowledge discovery and distribution process.

Abstract

Authoring survey or review articles still requires significant tedious manual effort, despite many advancements in research knowledge management having the potential to improve efficiency, reproducibility, and reuse. However, these advancements bring forth an increasing number of approaches, tools, and systems, which often cover only specific stages and lack a comprehensive workflow utilizing their task-specific strengths. We propose the Streamlined Workflow Automation for Machine-actionable Systematic Literature Reviews (SWARM-SLR) to crowdsource the improvement of SLR efficiency while maintaining scientific integrity in a state-of-the-art knowledge discovery and distribution process. The workflow aims to domain-independently support researchers in collaboratively and sustainably managing the rising scholarly knowledge corpus. By synthesizing guidelines from the literature, we have composed a set of 65 requirements, spanning from planning to reporting a review. Existing tools were assessed against these requirements and synthesized into the SWARM-SLR workflow prototype, a ready-for-operation software support tool. The SWARM-SLR was evaluated via two online surveys, which largely confirmed the validity of the 65 requirements and situated 11 tools to the different life-cycle stages. The SWARM-SLR workflow was similarly evaluated and found to be supporting almost the entire span of an SLR, excelling specifically in search and retrieval, information extraction, knowledge synthesis, and distribution. Our SWARM-SLR requirements and workflow support tool streamlines the SLR support for researchers, allowing sustainable collaboration by linking individual efficiency improvements to crowdsourced knowledge management. If these efforts are continued, we expect the increasing number of tools to be manageable and usable inside fully structured, (semi-)automated literature review workflows.

Figures (8)

• Figure 1: Formalizing SWARM-SLR requirements informs the identification and development of the tool catalog. Synthesizing these into a tool-assisted workflow makes this knowledge actionable and enables evaluation.
• Figure 2: The SWARM-SLR workflow enables and is enabled by the semantification of literature. It also produces a born-digital literature review, catalyzing future knowledge synthesis.
• Figure 3: SWARM-SLR workflow guided by four online Jupyter Notebooks.
• Figure 4: UML activity diagram for the fully automated Task 4: Select
• Figure 5: Obsidian overview of the analyzed library. The graph represents documents as nodes connected to similar documents and colored based on their relevance to a research question (RQ) (here: RQ1 red, RQ2 blue, average green). The RQ1 file on the left lists its dynamically ranked most relevant documents.