EVOKE: Elevating Chest X-ray Report Generation via Multi-View Contrastive Learning and Patient-Specific Knowledge

TL;DR

EVOKE tackles automated chest X-ray report generation by leveraging multi-view radiographs and patient INDICATION. It introduces a two-stage training framework: Stage 1 employs multi-view contrastive learning to align multiple views with report semantics, and Stage 2 performs knowledge-guided report generation using an INDICATION-aware transition bridge to handle missing indications. The approach achieves state-of-the-art results across MIMIC-CXR, MIMIC-ABN, Multi-view CXR, and Two-view CXR datasets, with significant improvements in RadGraph F1, BLEU, and CheXbert metrics, and demonstrates robustness through ablations and human evaluation. By providing curated Multi-view CXR and Two-view CXR datasets, EVOKE facilitates further research on multi-view radiology report generation and has potential to improve diagnostic efficiency and consistency in settings with diverse imaging views and available clinical indications.

Abstract

Radiology reports are crucial for planning treatment strategies and facilitating effective doctor-patient communication. However, the manual creation of these reports places a significant burden on radiologists. While automatic radiology report generation presents a promising solution, existing methods often rely on single-view radiographs, which constrain diagnostic accuracy. To address this challenge, we propose \textbf{EVOKE}, a novel chest X-ray report generation framework that incorporates multi-view contrastive learning and patient-specific knowledge. Specifically, we introduce a multi-view contrastive learning method that enhances visual representation by aligning multi-view radiographs with their corresponding report. After that, we present a knowledge-guided report generation module that integrates available patient-specific indications (e.g., symptom descriptions) to trigger the production of accurate and coherent radiology reports. To support research in multi-view report generation, we construct Multi-view CXR and Two-view CXR datasets using publicly available sources. Our proposed EVOKE surpasses recent state-of-the-art methods across multiple datasets, achieving a 2.9\% F\textsubscript{1} RadGraph improvement on MIMIC-CXR, a 7.3\% BLEU-1 improvement on MIMIC-ABN, a 3.1\% BLEU-4 improvement on Multi-view CXR, and an 8.2\% F\textsubscript{1,mic-14} CheXbert improvement on Two-view CXR.

Figures (4)

• Figure 1: A comparison between existing methods and our proposed approach for chest X-ray report generation reveals that existing methods rely on single-view images, whereas our approach leverages multi-view radiographs and patient-specific indications.
• Figure 2: Illustration of our proposed EVOKE, which comprises a visual encoder, a text encoder, and a text generator. EVOKE employs a two-stage training strategy: Multi-view contrastive learning for representation learning (Stage 1) and knowledge-guided report generation (Stage 2). The model inference is performed solely using Stage 2.
• Figure 3: Examples of generated reports on the MIMIC-CXR test set with a resolution of $224^2$. The cell "A/B" represents "EVOKE/R2Gen". Sentences in the reference report are shown in different colors. Each sentence in generated reports is highlighted in matching colors corresponding to those in the reference report. Failure sentences in EVOKE are underlined.
• Figure 4: Comparisons with baselines on the MIMIC-CXR test set, evaluated by "#Matched Findings" and GREEN score. "EVOKE-s" represents a resolution-specific variant of our proposed EVOKE framework.