Performance Evaluation of Transfer Learning Based Medical Image Classification Techniques for Disease Detection

TL;DR

This work assesses transfer learning for medical image classification by benchmarking six pre-trained CNNs (AlexNet, VGG16, ResNet-18/34/50, InceptionV3) on a chest X-ray dataset with normal, pneumonia, tuberculosis, and unknown classes. It employs a frozen feature extractor and a lightweight FFN classifier, evaluated under ImageNet pretraining with standard metrics (Precision, Recall, Accuracy, F1) plus uncertainty and runtime analyses. Results show InceptionV3 as the top performer across metrics, with deeper ResNets offering strong performance and AlexNet/VGG16 trailing, while computation cost scales with model depth. The findings guide model selection for TL in medical imaging, balancing accuracy, reliability, and resource constraints, and suggest avenues for faster inference in future work, with ViT architectures not included in this study.

Abstract

Medical image classification plays an increasingly vital role in identifying various diseases by classifying medical images, such as X-rays, MRIs and CT scans, into different categories based on their features. In recent years, deep learning techniques have attracted significant attention in medical image classification. However, it is usually infeasible to train an entire large deep learning model from scratch. To address this issue, one of the solutions is the transfer learning (TL) technique, where a pre-trained model is reused for a new task. In this paper, we present a comprehensive analysis of TL techniques for medical image classification using deep convolutional neural networks. We evaluate six pre-trained models (AlexNet, VGG16, ResNet18, ResNet34, ResNet50, and InceptionV3) on a custom chest X-ray dataset for disease detection. The experimental results demonstrate that InceptionV3 consistently outperforms other models across all the standard metrics. The ResNet family shows progressively better performance with increasing depth, whereas VGG16 and AlexNet perform reasonably well but with lower accuracy. In addition, we also conduct uncertainty analysis and runtime comparison to assess the robustness and computational efficiency of these models. Our findings reveal that TL is beneficial in most cases, especially with limited data, but the extent of improvement depends on several factors such as model architecture, dataset size, and domain similarity between source and target tasks. Moreover, we demonstrate that with a well-trained feature extractor, only a lightweight feedforward model is enough to provide efficient prediction. As such, this study contributes to the understanding of TL in medical image classification, and provides insights for selecting appropriate models based on specific requirements.

Figures (3)

• Figure 1: Schematic diagram of transfer learning
• Figure 2: Performance variations of various models
• Figure 3: Runtime comparison