Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Less Context, Same Performance: A RAG Framework for Resource-Efficient LLM-Based Clinical NLP

Satya Narayana Cheetirala, Ganesh Raut, Dhavalkumar Patel, Fabio Sanatana, Robert Freeman, Matthew A Levin, Girish N. Nadkarni, Omar Dawkins, Reba Miller, Randolph M. Steinhagen, Eyal Klang, Prem Timsina

TL;DR

The paper tackles the challenge of applying large language models to lengthy clinical notes, where token limits and costs hinder full-document processing. It proposes a Retrieval-Augmented Generation (RAG) framework that retrieves the most relevant text chunks via a FAISS index and feeds a compact, ≤4,000-token context to LLMs, comparing against whole-text ingestion across GPT4o, LLaMA, and Mistral for postoperative complication classification. Results show that RAG achieves comparable AUROC and F1 to long-context processing while dramatically reducing token usage (over 90%) and costs, with DeLong tests indicating no significant performance loss. This demonstrates that RAG is a scalable, model-agnostic approach that can extend to broader clinical NLP tasks while maintaining accuracy and reducing computational burden.

Abstract

Long text classification is challenging for Large Language Models (LLMs) due to token limits and high computational costs. This study explores whether a Retrieval Augmented Generation (RAG) approach using only the most relevant text segments can match the performance of processing entire clinical notes with large context LLMs. We begin by splitting clinical documents into smaller chunks, converting them into vector embeddings, and storing these in a FAISS index. We then retrieve the top 4,000 words most pertinent to the classification query and feed these consolidated segments into an LLM. We evaluated three LLMs (GPT4o, LLaMA, and Mistral) on a surgical complication identification task. Metrics such as AUC ROC, precision, recall, and F1 showed no statistically significant differences between the RAG based approach and whole-text processing (p > 0.05p > 0.05). These findings indicate that RAG can significantly reduce token usage without sacrificing classification accuracy, providing a scalable and cost effective solution for analyzing lengthy clinical documents.

Less Context, Same Performance: A RAG Framework for Resource-Efficient LLM-Based Clinical NLP

TL;DR

The paper tackles the challenge of applying large language models to lengthy clinical notes, where token limits and costs hinder full-document processing. It proposes a Retrieval-Augmented Generation (RAG) framework that retrieves the most relevant text chunks via a FAISS index and feeds a compact, ≤4,000-token context to LLMs, comparing against whole-text ingestion across GPT4o, LLaMA, and Mistral for postoperative complication classification. Results show that RAG achieves comparable AUROC and F1 to long-context processing while dramatically reducing token usage (over 90%) and costs, with DeLong tests indicating no significant performance loss. This demonstrates that RAG is a scalable, model-agnostic approach that can extend to broader clinical NLP tasks while maintaining accuracy and reducing computational burden.

Abstract

Long text classification is challenging for Large Language Models (LLMs) due to token limits and high computational costs. This study explores whether a Retrieval Augmented Generation (RAG) approach using only the most relevant text segments can match the performance of processing entire clinical notes with large context LLMs. We begin by splitting clinical documents into smaller chunks, converting them into vector embeddings, and storing these in a FAISS index. We then retrieve the top 4,000 words most pertinent to the classification query and feed these consolidated segments into an LLM. We evaluated three LLMs (GPT4o, LLaMA, and Mistral) on a surgical complication identification task. Metrics such as AUC ROC, precision, recall, and F1 showed no statistically significant differences between the RAG based approach and whole-text processing (p > 0.05p > 0.05). These findings indicate that RAG can significantly reduce token usage without sacrificing classification accuracy, providing a scalable and cost effective solution for analyzing lengthy clinical documents.

Paper Structure

This paper contains 35 sections, 8 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Token Constraints and Computational Costs
  4. Retrieval-Augmented Generation (RAG)
  5. Chunking vs. Whole-Text Ingestion
  6. Chunking Approaches
  7. Whole-Text Ingestion
  8. Vector Embeddings
  9. Implementing Top-N Retrieval with FAISS
  10. Methodology
  11. Dataset and Labeling
  12. Study Design
  13. A1-Long Context Approach
  14. A2-RAG Approach
  15. Vector Database Creation
  16. ...and 20 more sections

Figures (8)

  • Figure 1: Overview of the study cohort. We randomly selected 996 cases with surgical complications (2013--2023) and 1,298 cases with no identified complications.
  • Figure 2: Overall study design comparing a long-context approach vs. a RAG approach for classifying post-operative complications.
  • Figure 3: Long Context Approach. Entire text from the past 30 days of clinical notes is concatenated and fed to the LLM in a single pass.
  • Figure 4: Vector database creation using FAISS. Clinical notes are chunked, embedded, and indexed for efficient similarity searches.
  • Figure 5: RAG-based classification. The top-$N$ chunks from the FAISS index are concatenated (under 4,000 tokens) and fed into an LLM classifier.
  • ...and 3 more figures