Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Prompt-Guided Generation of Structured Chest X-Ray Report Using a Pre-trained LLM

Hongzhao Li, Hongyu Wang, Xia Sun, Hua He, Jun Feng

TL;DR

The paper tackles unstructured and non-interactive chest X-ray report generation by introducing a prompt-guided framework that leverages anatomy-region detection to create region-specific sentences, then guides a pre-trained LLM with anatomy and clinical context prompts to produce structured reports. Key contributions include an anatomy-focused detection and sentence-generation pipeline, a binary-prompt system for region presence and abnormalities, and GPT-4 integration with clinician-provided context to yield clinically accurate, interpretable reports. Evaluations on MIMIC-CXR with standard NLG metrics and a clinical-efficiency measure show superior language quality and improved clinical alignment compared to prior methods, along with strong region-level localization. The approach enhances interpretability and interactivity, enabling physician input to tailor reports to patient context and examination rationale, potentially reducing workload and diagnostic error in radiology.

Abstract

Medical report generation automates radiology descriptions from images, easing the burden on physicians and minimizing errors. However, current methods lack structured outputs and physician interactivity for clear, clinically relevant reports. Our method introduces a prompt-guided approach to generate structured chest X-ray reports using a pre-trained large language model (LLM). First, we identify anatomical regions in chest X-rays to generate focused sentences that center on key visual elements, thereby establishing a structured report foundation with anatomy-based sentences. We also convert the detected anatomy into textual prompts conveying anatomical comprehension to the LLM. Additionally, the clinical context prompts guide the LLM to emphasize interactivity and clinical requirements. By integrating anatomy-focused sentences and anatomy/clinical prompts, the pre-trained LLM can generate structured chest X-ray reports tailored to prompted anatomical regions and clinical contexts. We evaluate using language generation and clinical effectiveness metrics, demonstrating strong performance.

Prompt-Guided Generation of Structured Chest X-Ray Report Using a Pre-trained LLM

TL;DR

The paper tackles unstructured and non-interactive chest X-ray report generation by introducing a prompt-guided framework that leverages anatomy-region detection to create region-specific sentences, then guides a pre-trained LLM with anatomy and clinical context prompts to produce structured reports. Key contributions include an anatomy-focused detection and sentence-generation pipeline, a binary-prompt system for region presence and abnormalities, and GPT-4 integration with clinician-provided context to yield clinically accurate, interpretable reports. Evaluations on MIMIC-CXR with standard NLG metrics and a clinical-efficiency measure show superior language quality and improved clinical alignment compared to prior methods, along with strong region-level localization. The approach enhances interpretability and interactivity, enabling physician input to tailor reports to patient context and examination rationale, potentially reducing workload and diagnostic error in radiology.

Abstract

Medical report generation automates radiology descriptions from images, easing the burden on physicians and minimizing errors. However, current methods lack structured outputs and physician interactivity for clear, clinically relevant reports. Our method introduces a prompt-guided approach to generate structured chest X-ray reports using a pre-trained large language model (LLM). First, we identify anatomical regions in chest X-rays to generate focused sentences that center on key visual elements, thereby establishing a structured report foundation with anatomy-based sentences. We also convert the detected anatomy into textual prompts conveying anatomical comprehension to the LLM. Additionally, the clinical context prompts guide the LLM to emphasize interactivity and clinical requirements. By integrating anatomy-focused sentences and anatomy/clinical prompts, the pre-trained LLM can generate structured chest X-ray reports tailored to prompted anatomical regions and clinical contexts. We evaluate using language generation and clinical effectiveness metrics, demonstrating strong performance.
Paper Structure (13 sections, 4 equations, 4 figures, 4 tables)

This paper contains 13 sections, 4 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Methodology
  4. Anatomy Region Detection and Feature Extraction
  5. Sentence Generator
  6. Anatomy Prompts Generation
  7. Structured report generation
  8. Experiments
  9. Datasets and Metrics
  10. Implementation Details
  11. Quantitative results
  12. Qualitative results
  13. Conclusion

Figures (4)

  • Figure 1: The high-level representation of our architecture. Each sentence can be explicitly associated with a specific region.
  • Figure 2: In the architecture overview, the process includes identifying and extracting anatomical region features, generating region descriptions, and ultimately integrating anatomical prompts with clinical context prompts to produce a structured report.
  • Figure 3: An example is presented to illustrate the inputs for the LLM.
  • Figure 4: We represented the detected anatomical regions, the corresponding generated sentences, and the semantically matched reference sentences using the same color, while also highlighting the clinical context.