Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

QUB-Cirdan at "Discharge Me!": Zero shot discharge letter generation by open-source LLM

Rui Guo, Greg Farnan, Niall McLaughlin, Barry Devereux

TL;DR

This paper presents the approach using the Llama3 8B quantized model to generate the “Brief Hospital Course” and “Discharge Instructions” sections, employing a zero-shot method combined with Retrieval-Augmented Generation (RAG) to produce concise, contextually accurate summaries.

Abstract

The BioNLP ACL'24 Shared Task on Streamlining Discharge Documentation aims to reduce the administrative burden on clinicians by automating the creation of critical sections of patient discharge letters. This paper presents our approach using the Llama3 8B quantized model to generate the "Brief Hospital Course" and "Discharge Instructions" sections. We employ a zero-shot method combined with Retrieval-Augmented Generation (RAG) to produce concise, contextually accurate summaries. Our contributions include the development of a curated template-based approach to ensure reliability and consistency, as well as the integration of RAG for word count prediction. We also describe several unsuccessful experiments to provide insights into our pathway for the competition. Our results demonstrate the effectiveness and efficiency of our approach, achieving high scores across multiple evaluation metrics.

QUB-Cirdan at "Discharge Me!": Zero shot discharge letter generation by open-source LLM

TL;DR

This paper presents the approach using the Llama3 8B quantized model to generate the “Brief Hospital Course” and “Discharge Instructions” sections, employing a zero-shot method combined with Retrieval-Augmented Generation (RAG) to produce concise, contextually accurate summaries.

Abstract

The BioNLP ACL'24 Shared Task on Streamlining Discharge Documentation aims to reduce the administrative burden on clinicians by automating the creation of critical sections of patient discharge letters. This paper presents our approach using the Llama3 8B quantized model to generate the "Brief Hospital Course" and "Discharge Instructions" sections. We employ a zero-shot method combined with Retrieval-Augmented Generation (RAG) to produce concise, contextually accurate summaries. Our contributions include the development of a curated template-based approach to ensure reliability and consistency, as well as the integration of RAG for word count prediction. We also describe several unsuccessful experiments to provide insights into our pathway for the competition. Our results demonstrate the effectiveness and efficiency of our approach, achieving high scores across multiple evaluation metrics.
Paper Structure (11 sections, 4 figures, 4 tables)

This paper contains 11 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Dataset Exploration
  5. Retrieval for the Target Section Word Count
  6. Target Section Structure Template and Prompt Creation
  7. Results
  8. Unsuccessful Attempts
  9. Conclusion
  10. Limitations and Future Work
  11. Ethical Statement

Figures (4)

  • Figure 1: Overview of our solution. The figure illustrates our four-step approach: (1) Text Segmentation: splitting the discharge letter into sections such as "Chief Complaint" and "Brief Hospital Course"; (2) Retrieval-Augmented Generation (RAG): retrieving similar patient sections to determine word count; (3) Template and Prompt Design: providing structured templates and prompts to Llama3 with patient context and target word count; (4) Text Generation: generating the final output using Llama3.
  • Figure 2: The target section word count distribution. Both BHC and DI have right-skewed distributions. BHC has two peaks, one below 100 words and one around 250 words.
  • Figure 3: The top 10 features for the BHC classifier. WC: word count. The total number of lab tests, diagnosis, and total duration in the hospital are the top 3 features.
  • Figure 4: The top 10 features for the DI classifier. WC: word count. The word count of different segments is ranking high.