DenseReviewer: A Screening Prioritisation Tool for Systematic Review based on Dense Retrieval
Xinyu Mao, Teerapong Leelanupab, Harrisen Scells, Guido Zuccon
TL;DR
The paper addresses the high workload of title and abstract screening in medical systematic reviews by introducing DenseReviewer, a screening tool that uses dense retrieval guided by PICO queries. A web-based interface and a Python library enable end users to screen studies and to prototype new active learning approaches, with feedback updating rankings via Rocchio's algorithm. DenseReviewer builds an end-to-end, Dockerized architecture (front end, API, database, task services) and supports training and experimentation with multiple dense retriever backbones. Future work aims to extend exclusion criteria, highlight evidence for PICO elements, and explore LLM-assisted relevance judgments to further reduce screening workload.
Abstract
Screening is a time-consuming and labour-intensive yet required task for medical systematic reviews, as tens of thousands of studies often need to be screened. Prioritising relevant studies to be screened allows downstream systematic review creation tasks to start earlier and save time. In previous work, we developed a dense retrieval method to prioritise relevant studies with reviewer feedback during the title and abstract screening stage. Our method outperforms previous active learning methods in both effectiveness and efficiency. In this demo, we extend this prior work by creating (1) a web-based screening tool that enables end-users to screen studies exploiting state-of-the-art methods and (2) a Python library that integrates models and feedback mechanisms and allows researchers to develop and demonstrate new active learning methods. We describe the tool's design and showcase how it can aid screening. The tool is available at https://densereviewer.ielab.io. The source code is also open sourced at https://github.com/ielab/densereviewer.