Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Expert Insight-Enhanced Follow-up Chest X-Ray Summary Generation

Zhichuan Wang, Kinhei Lee, Qiao Deng, Tiffany Y. So, Wan Hang Chiu, Yeung Yu Hui, Bingjing Zhou, Edward S. Hui

TL;DR

The paper tackles automatic follow-up chest X-ray summary generation, a clinically important problem that requires describing disease progression and device changes across paired X-ray exams. It introduces a transformer-based Expert Insight-Enhanced (EIE) framework that couples Expert Soft Guidance with a Masked Entity Modeling Loss to improve fidelity for abnormality-related words, using an Expert Guided Difference Capture Module (EGDCM) and a cross-modality summary generator. Across the MIMIC-Diff-VQA dataset, EIE variants outperform state-of-the-art methods, with EIE-all delivering the largest gains on both generation metrics (e.g., BLEU, METEOR, ROUGE-L, CIDEr) and clinical accuracy (Acc5, Acc14). The work highlights the robustness of soft guidance over hard thresholds and demonstrates the value of integrating domain-specific lexical cues into radiology report generation, though it notes limitations in clinical localization and calls for future work to predict abnormality locations.

Abstract

A chest X-ray radiology report describes abnormal findings not only from X-ray obtained at current examination, but also findings on disease progression or change in device placement with reference to the X-ray from previous examination. Majority of the efforts on automatic generation of radiology report pertain to reporting the former, but not the latter, type of findings. To the best of the authors' knowledge, there is only one work dedicated to generating summary of the latter findings, i.e., follow-up summary. In this study, we therefore propose a transformer-based framework to tackle this task. Motivated by our observations on the significance of medical lexicon on the fidelity of summary generation, we introduce two mechanisms to bestow expert insight to our model, namely expert soft guidance and masked entity modeling loss. The former mechanism employs a pretrained expert disease classifier to guide the presence level of specific abnormalities, while the latter directs the model's attention toward medical lexicon. Extensive experiments were conducted to demonstrate that the performance of our model is competitive with or exceeds the state-of-the-art.

Expert Insight-Enhanced Follow-up Chest X-Ray Summary Generation

TL;DR

The paper tackles automatic follow-up chest X-ray summary generation, a clinically important problem that requires describing disease progression and device changes across paired X-ray exams. It introduces a transformer-based Expert Insight-Enhanced (EIE) framework that couples Expert Soft Guidance with a Masked Entity Modeling Loss to improve fidelity for abnormality-related words, using an Expert Guided Difference Capture Module (EGDCM) and a cross-modality summary generator. Across the MIMIC-Diff-VQA dataset, EIE variants outperform state-of-the-art methods, with EIE-all delivering the largest gains on both generation metrics (e.g., BLEU, METEOR, ROUGE-L, CIDEr) and clinical accuracy (Acc5, Acc14). The work highlights the robustness of soft guidance over hard thresholds and demonstrates the value of integrating domain-specific lexical cues into radiology report generation, though it notes limitations in clinical localization and calls for future work to predict abnormality locations.

Abstract

A chest X-ray radiology report describes abnormal findings not only from X-ray obtained at current examination, but also findings on disease progression or change in device placement with reference to the X-ray from previous examination. Majority of the efforts on automatic generation of radiology report pertain to reporting the former, but not the latter, type of findings. To the best of the authors' knowledge, there is only one work dedicated to generating summary of the latter findings, i.e., follow-up summary. In this study, we therefore propose a transformer-based framework to tackle this task. Motivated by our observations on the significance of medical lexicon on the fidelity of summary generation, we introduce two mechanisms to bestow expert insight to our model, namely expert soft guidance and masked entity modeling loss. The former mechanism employs a pretrained expert disease classifier to guide the presence level of specific abnormalities, while the latter directs the model's attention toward medical lexicon. Extensive experiments were conducted to demonstrate that the performance of our model is competitive with or exceeds the state-of-the-art.
Paper Structure (22 sections, 7 equations, 3 figures, 2 tables)

This paper contains 22 sections, 7 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Preliminaries
  5. Model Architecture
  6. Expert Soft Guidance
  7. Expert-Guided Difference Capture Module
  8. Cross-Modality Follow-Up Summary Generator
  9. Loss Functions
  10. Masked Language Modeling Loss
  11. Masked Entity Modeling Loss
  12. Training Strategy
  13. Experiments
  14. Dataset and Metrics
  15. Implementation Details
  16. ...and 7 more sections

Figures (3)

  • Figure 1: A schematic of our proposed model that generates a textural summary of disease progression from a pair of chest X-rays taken at follow-up and baseline examinations.
  • Figure 2: Illustration of our proposed framework, expert insight-enhanced (EIE) follow-up chest X-ray summary generation. The dotted arrows indicate the branches that are only enabled in the training phase, and the symbol of lock indicates module that are frozen. EGDCM: expert guided difference capture module; MLM: masked language modelling; MEM: masked entity modelling.
  • Figure 3: Hyperparameter $\alpha$ sensitivity study of EIE-mem.