Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IgnitionInnovators at "Discharge Me!": Chain-of-Thought Instruction Finetuning Large Language Models for Discharge Summaries

An Quang Tang, Xiuzhen Zhang, Minh Ngoc Dinh

TL;DR

The paper introduces Discharge-LLM, a framework for automatically generating two critical discharge-summary sections (Brief Hospital Course and Discharge Instructions) by instruction-finetuning a lightweight Mistral LLM with LoRA adapters and Chain-of-Thought prompting. It integrates three stages—Section Extraction, Radiology Report Selection, and Target Section Generation—and employs five-part prompts with CoT questions to improve structure and factual fidelity. Key innovations include selective extraction of 13 relevant sections, substitution with radiology reports to reduce noise, and per-task adapters tailored to DSD. Experimental results on MIMIC-IV data show meaningful gains from contextual prompting and CoT, though there remains a gap to the best-performing systems in the shared task, suggesting that larger-scale data and further refinements could close the gap while enabling privacy-preserving, local deployment in clinical settings.

Abstract

This paper presents our proposed approach to the Discharge Me! shared task, collocated with the 23th Workshop on Biomedical Natural Language Processing (BioNLP). In this work, we develop an LLM-based framework for solving the Discharge Summary Documentation (DSD) task, i.e., generating the two critical target sections `Brief Hospital Course' and `Discharge Instructions' in the discharge summary. By streamlining the recent instruction-finetuning process on LLMs, we explore several prompting strategies for optimally adapting LLMs to specific generation task of DSD. Experimental results show that providing a clear output structure, complimented by a set of comprehensive Chain-of-Thoughts (CoT) questions, effectively improves the model's reasoning capability, and thereby, enhancing the structural correctness and faithfulness of clinical information in the generated text. Source code is available at: https://github.com/antangrocket1312/Discharge_LLM

IgnitionInnovators at "Discharge Me!": Chain-of-Thought Instruction Finetuning Large Language Models for Discharge Summaries

TL;DR

The paper introduces Discharge-LLM, a framework for automatically generating two critical discharge-summary sections (Brief Hospital Course and Discharge Instructions) by instruction-finetuning a lightweight Mistral LLM with LoRA adapters and Chain-of-Thought prompting. It integrates three stages—Section Extraction, Radiology Report Selection, and Target Section Generation—and employs five-part prompts with CoT questions to improve structure and factual fidelity. Key innovations include selective extraction of 13 relevant sections, substitution with radiology reports to reduce noise, and per-task adapters tailored to DSD. Experimental results on MIMIC-IV data show meaningful gains from contextual prompting and CoT, though there remains a gap to the best-performing systems in the shared task, suggesting that larger-scale data and further refinements could close the gap while enabling privacy-preserving, local deployment in clinical settings.

Abstract

This paper presents our proposed approach to the Discharge Me! shared task, collocated with the 23th Workshop on Biomedical Natural Language Processing (BioNLP). In this work, we develop an LLM-based framework for solving the Discharge Summary Documentation (DSD) task, i.e., generating the two critical target sections `Brief Hospital Course' and `Discharge Instructions' in the discharge summary. By streamlining the recent instruction-finetuning process on LLMs, we explore several prompting strategies for optimally adapting LLMs to specific generation task of DSD. Experimental results show that providing a clear output structure, complimented by a set of comprehensive Chain-of-Thoughts (CoT) questions, effectively improves the model's reasoning capability, and thereby, enhancing the structural correctness and faithfulness of clinical information in the generated text. Source code is available at: https://github.com/antangrocket1312/Discharge_LLM
Paper Structure (22 sections, 2 figures, 7 tables)

This paper contains 22 sections, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem description
  4. Methodology
  5. Section Extraction
  6. Radiology Report Selection
  7. Target Section Generation
  8. Prompting Strategies
  9. Experiments
  10. Baseline and Implementation Details
  11. Metrics
  12. Dataset
  13. Data Preprocessing
  14. Results
  15. Shared Task's Evaluation Results
  16. ...and 7 more sections

Figures (2)

  • Figure 1: The Discharge-LLM framework
  • Figure 2: Distribution of samples per number of words on phase 2's test set