ICON: Improving Inter-Report Consistency in Radiology Report Generation via Lesion-aware Mixup Augmentation

TL;DR

The paper tackles inter-report consistency in radiology report generation by introducing ICon, a lesion-aware, two-stage framework that first extracts lesions (Zoomer) and then generates consistent reports through lesion-attribute alignment (Inspector) and a cross-attentive generator (FiD/BART). A lesion-aware mixup augments training to align semantically equivalent lesions, while two metrics, Con and R-Con, quantify inter-report consistency with reference-quality weighting. Extensive experiments on IU X-ray, MIMIC-CXR, and MIMIC-ABN demonstrate that ICon achieves state-of-the-art inter-report consistency and competitive clinical accuracy, highlighting the value of region-level lesion reasoning for trustworthy radiology narration. The work suggests practical impact in improving credibility and robustness of automated radiology reporting, with future directions including incorporating large language models and end-to-end optimization ideas to further enhance performance.

Abstract

Previous research on radiology report generation has made significant progress in terms of increasing the clinical accuracy of generated reports. In this paper, we emphasize another crucial quality that it should possess, i.e., inter-report consistency, which refers to the capability of generating consistent reports for semantically equivalent radiographs. This quality is even of greater significance than the overall report accuracy in terms of ensuring the system's credibility, as a system prone to providing conflicting results would severely erode users' trust. Regrettably, existing approaches struggle to maintain inter-report consistency, exhibiting biases towards common patterns and susceptibility to lesion variants. To address this issue, we propose ICON, which improves the inter-report consistency of radiology report generation. Aiming to enhance the system's ability to capture similarities in semantically equivalent lesions, our approach first involves extracting lesions from input images and examining their characteristics. Then, we introduce a lesion-aware mixup technique to ensure that the representations of the semantically equivalent lesions align with the same attributes, achieved through a linear combination during the training phase. Extensive experiments on three publicly available chest X-ray datasets verify the effectiveness of our approach, both in terms of improving the consistency and accuracy of the generated reports.

Figures (5)

• Figure 1: Given two semantically equivalent cases (i.e., Case A and Case B), an example to illustrate the difference between three radiology report generation systems: a consistent and accurate system (i.e., System $\alpha$) and a consistently inaccurate system (i.e., System $\beta$), and an inconsistent system (i.e., System $\gamma$).
• Figure 2: Overview of the ICon framework, which first extracts lesions and then generates reports. Attributes are extracted from RadGraph radgraph.
• Figure 3: Overview of our proposed lesion-aware mixup augmentation.
• Figure 4: A case study of ICon on two semantically equivalent cases (i.e., Case A and Case B), given their radiographs and lesions. Spans with the same color (Cardiomegaly, Pleural Effusion, Atelectasis, and Edema) represent the same positive observation. Consistent and accurate outputs are highlighted with underline.
• Figure 5: An error case produced by ICon. The span and the span denote false negative and false positive observations, respectively.