RadBARTsum: Domain Specific Adaption of Denoising Sequence-to-Sequence Models for Abstractive Radiology Report Summarization

TL;DR

RadBARTsum presents a domain-adaptive approach to radiology report summarization by first re-training a BART model with medical entity masking to embed clinical knowledge, followed by task-specific fine-tuning to predict the Impression from Findings and Background. The method demonstrates improvements in ROUGE-L and BertScore on MIMIC-CXR and Stanford datasets, with word-level entity masking yielding the strongest gains. The work advances radiology NLP by injecting domain knowledge to reduce hallucinations and produce more clinically accurate summaries, potentially speeding clinical decision-making. Limitations include tokenizer adaptation and hallucination mitigation, with future work exploring ontologies and human evaluation to further enhance performance and reliability.

Abstract

Radiology report summarization is a crucial task that can help doctors quickly identify clinically significant findings without the need to review detailed sections of reports. This study proposes RadBARTsum, a domain-specific and ontology facilitated adaptation of the BART model for abstractive radiology report summarization. The approach involves two main steps: 1) re-training the BART model on a large corpus of radiology reports using a novel entity masking strategy to improving biomedical domain knowledge learning, and 2) fine-tuning the model for the summarization task using the Findings and Background sections to predict the Impression section. Experiments are conducted using different masking strategies. Results show that the re-training process with domain knowledge facilitated masking improves performances consistently across various settings. This work contributes a domain-specific generative language model for radiology report summarization and a method for utilising medical knowledge to realise entity masking language model. The proposed approach demonstrates a promising direction of enhancing the efficiency of language models by deepening its understanding of clinical knowledge in radiology reports.

Figures (3)

• Figure 1: Architecture of the medical entity masking.
• Figure 2: Our proposed work compared with the previous work.
• Figure 3: The effect of the proportion of medical entity masking on the summarization task performance.