Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLAssist: Simple Tools for Automating Literature Review Using Large Language Models

Christoforus Yoga Haryanto

TL;DR

LLAssist tackles the challenge of exponential growth in scholarly publications by providing an open-source tool that automates key stages of literature screening using Large Language Models. The system pipelines data input, key semantics extraction, relevance and contribution estimation, must-read prioritization, and output generation into JSON/CSV outputs, enabling transparent human-in-the-loop review. It introduces a novel LLM-based relevance estimation and compares multiple backends (e.g., Gemma 2, Llama 3, GPT-3.5, GPT-4o) to assess performance across domains, with a threshold-based scoring scheme. The work demonstrates substantial time savings and improved consistency in initial screening while acknowledging limitations such as reliance on LLM quality and the need for full-text analysis and domain-specific tuning. Overall, LLAssist contributes to open, reproducible AI-assisted literature reviews and offers a practical building block for scalable knowledge bases.

Abstract

This paper introduces LLAssist, an open-source tool designed to streamline literature reviews in academic research. In an era of exponential growth in scientific publications, researchers face mounting challenges in efficiently processing vast volumes of literature. LLAssist addresses this issue by leveraging Large Language Models (LLMs) and Natural Language Processing (NLP) techniques to automate key aspects of the review process. Specifically, it extracts important information from research articles and evaluates their relevance to user-defined research questions. The goal of LLAssist is to significantly reduce the time and effort required for comprehensive literature reviews, allowing researchers to focus more on analyzing and synthesizing information rather than on initial screening tasks. By automating parts of the literature review workflow, LLAssist aims to help researchers manage the growing volume of academic publications more efficiently.

LLAssist: Simple Tools for Automating Literature Review Using Large Language Models

TL;DR

LLAssist tackles the challenge of exponential growth in scholarly publications by providing an open-source tool that automates key stages of literature screening using Large Language Models. The system pipelines data input, key semantics extraction, relevance and contribution estimation, must-read prioritization, and output generation into JSON/CSV outputs, enabling transparent human-in-the-loop review. It introduces a novel LLM-based relevance estimation and compares multiple backends (e.g., Gemma 2, Llama 3, GPT-3.5, GPT-4o) to assess performance across domains, with a threshold-based scoring scheme. The work demonstrates substantial time savings and improved consistency in initial screening while acknowledging limitations such as reliance on LLM quality and the need for full-text analysis and domain-specific tuning. Overall, LLAssist contributes to open, reproducible AI-assisted literature reviews and offers a practical building block for scalable knowledge bases.

Abstract

This paper introduces LLAssist, an open-source tool designed to streamline literature reviews in academic research. In an era of exponential growth in scientific publications, researchers face mounting challenges in efficiently processing vast volumes of literature. LLAssist addresses this issue by leveraging Large Language Models (LLMs) and Natural Language Processing (NLP) techniques to automate key aspects of the review process. Specifically, it extracts important information from research articles and evaluates their relevance to user-defined research questions. The goal of LLAssist is to significantly reduce the time and effort required for comprehensive literature reviews, allowing researchers to focus more on analyzing and synthesizing information rather than on initial screening tasks. By automating parts of the literature review workflow, LLAssist aims to help researchers manage the growing volume of academic publications more efficiently.
Paper Structure (30 sections, 3 figures, 1 table)

This paper contains 30 sections, 3 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. METHODOLOGY
  3. LLAssist: An LLM-based Literature Screening Tool
  4. Data Input
  5. Article Processing
  6. Key Semantics Extraction
  7. Relevance Estimation
  8. Must-Read Determination
  9. Output Generation
  10. Experimental Evaluation
  11. Data Collection
  12. Research Questions
  13. Automated Analysis
  14. Evaluation Metrics
  15. Preliminary Nature of Evaluation
  16. ...and 15 more sections

Figures (3)

  • Figure 1: Sequence diagram of LLAssist console application
  • Figure 2: Binary Relevance and Relevance Score Distribution (small datasets)
  • Figure 3: Must-read vs. Discard ratio (all datasets)