Multi-domain improves out-of-distribution and data-limited scenarios for medical image analysis

TL;DR

This paper addresses the challenge of generalization in medical image analysis when data are scarce or distribution shifts occur. It proposes a multi-domain approach that trains on diverse imaging modalities and viewpoints, using an off-the-shelf architecture to perform a single task across domains. Across PolyMNIST, MedMNIST, and ImageCLEFmedical, the multi-domain model consistently improves out-of-distribution and data-limited performance while maintaining or improving in-distribution accuracy, with gains up to about $8\%$ in organ-related tasks. The findings suggest that leveraging cross-domain information can significantly enhance robustness of medical image analysis systems in real-world, data-constrained healthcare settings.

Abstract

Current machine learning methods for medical image analysis primarily focus on developing models tailored for their specific tasks, utilizing data within their target domain. These specialized models tend to be data-hungry and often exhibit limitations in generalizing to out-of-distribution samples. In this work, we show that employing models that incorporate multiple domains instead of specialized ones significantly alleviates the limitations observed in specialized models. We refer to this approach as multi-domain model and compare its performance to that of specialized models. For this, we introduce the incorporation of diverse medical image domains, including different imaging modalities like X-ray, MRI, CT, and ultrasound images, as well as various viewpoints such as axial, coronal, and sagittal views. Our findings underscore the superior generalization capabilities of multi-domain models, particularly in scenarios characterized by limited data availability and out-of-distribution, frequently encountered in healthcare applications. The integration of diverse data allows multi-domain models to utilize information across domains, enhancing the overall outcomes substantially. To illustrate, for organ recognition, multi-domain model can enhance accuracy by up to 8% compared to conventional specialized models.

Figures (20)

• Figure 1: We employ (a) PolyMNIST, multi-modal dataset for digit classification using data from different modalities, (b) MedMNIST for the classification of organs from different views of CT image slices, and (c) ImageCLEFmedical for organ classification using data from different imaging modalities.
• Figure 2: Number of images for medical datasets. (a) For MedMNIST: (i) training, (ii) validation, and (iii) test set. (b) For ImageCLEFmedical: (i) training and validation, and (ii) test set.
• Figure 3: Data diversity evaluation for PolyMNIST: (a) normalized digit distribution using $\mu_\text{digit}=0$ and $\sigma_\text{digit}=5$, (b) normalized modality distribution with $\mu_\text{modality}=2$ and $\sigma_\text{modality}=3$, (c) number of samples from the resulting distribution for each digit/modality combination, (d) example of data distribution and OOD scenario with a 100% OOD level for digit 2 and modality d.
• Figure 4: Training and evaluation schemes for specialized and multi-domain models: (a,d) represent the training data utilized by specialized and multi-domain models using $\mu_\text{digit}=0$, $\sigma_\text{digit}=5$, $\mu_\text{modality}=2$ and $\sigma_\text{modality}=3$ with a 100% OOD level for digit 2 and modality d. (b,e) show the OOD evaluation specifically for digit 2 and modality d. (c,f) demonstrate the ID evaluation for all other digit/modality combinations except digit 2 and modality d.
• Figure 5: Evaluating OOD levels for PolyMNIST. Each point shows the area under the balanced accuracy curve through different evaluation of data distributions for different OOD levels in x-axis.