R2Gen-Mamba: A Selective State Space Model for Radiology Report Generation

TL;DR

R2Gen-Mamba is presented, a novel automatic radiology report generation method that leverages the efficient sequence processing of the Mamba with the contextual benefits of Transformer architectures that not only enhances training and inference efficiency but also produces high-quality reports.

Abstract

Radiology report generation is crucial in medical imaging,but the manual annotation process by physicians is time-consuming and labor-intensive, necessitating the develop-ment of automatic report generation methods. Existingresearch predominantly utilizes Transformers to generateradiology reports, which can be computationally intensive,limiting their use in real applications. In this work, we presentR2Gen-Mamba, a novel automatic radiology report genera-tion method that leverages the efficient sequence processingof the Mamba with the contextual benefits of Transformerarchitectures. Due to lower computational complexity ofMamba, R2Gen-Mamba not only enhances training and in-ference efficiency but also produces high-quality reports.Experimental results on two benchmark datasets with morethan 210,000 X-ray image-report pairs demonstrate the ef-fectiveness of R2Gen-Mamba regarding report quality andcomputational efficiency compared with several state-of-the-art methods. The source code can be accessed online.

Figures (2)

• Figure 1: Architecture of the proposed R2Gen-Mamba framework, with visual extractor and decoder denoted by gray dashed boxes. The Mamba encoder is highlighted within green dashed boxes. Conv: convolution; SSM: selective state space model; Linear: linear projection.
• Figure 2: Examples of ground truth and generated reports by different methods, with similar findings marked in the same color.