Less is More: Selective Reduction of CT Data for Self-Supervised Pre-Training of Deep Learning Models with Contrastive Learning Improves Downstream Classification Performance

Abstract

Self-supervised pre-training of deep learning models with contrastive learning is a widely used technique in image analysis. Current findings indicate a strong potential for contrastive pre-training on medical images. However, further research is necessary to incorporate the particular characteristics of these images. We hypothesize that the similarity of medical images hinders the success of contrastive learning in the medical imaging domain. To this end, we investigate different strategies based on deep embedding, information theory, and hashing in order to identify and reduce redundancy in medical pre-training datasets. The effect of these different reduction strategies on contrastive learning is evaluated on two pre-training datasets and several downstream classification tasks. In all of our experiments, dataset reduction leads to a considerable performance gain in downstream tasks, e.g., an AUC score improvement from 0.78 to 0.83 for the COVID CT Classification Grand Challenge, 0.97 to 0.98 for the OrganSMNIST Classification Challenge and 0.73 to 0.83 for a brain hemorrhage classification task. Furthermore, pre-training is up to nine times faster due to the dataset reduction. In conclusion, the proposed approach highlights the importance of dataset quality and provides a transferable approach to improve contrastive pre-training for classification downstream tasks on medical images.

Figures (12)

• Figure 1: This figure gives an overview of our approach to investigate the hypothesis that using all CT slices for contrastive pre-training may lead to performance degradation due to the high similarity of the slices. The first step is to pre-train a deep learning model. Therefore, we start with a dataset of unannotated CT slices, select slices in such a way that we obtain a reduced dataset with increased variation, and pre-train the model with contrastive learning on the reduced dataset. The second step is to evaluate the pre-training on downstream tasks. Therefore, the pre-trained model is fine-tuned with supervised learning on small datasets with annotations for the specific task. Our work compares different strategies to reduce CT image pre-training datasets.
• Figure 2: This figure explains the similarity calculation between two images using the HASH method. First, the images are compressed to the size size of $9\times8$. In the second step, a 64-bit hash is computed by looping through each row of the compressed images, comparing each pixel with its right neighbor, and choosing one if the neighbor is larger and zero if the neighbor is smaller. To calculate the similarity, the hashes for the two images are compared with the Hamming distance, which is the sum of the different bits.
• Figure 3: Example slices of the COVID-19 classification downstream task from Grand Challenge yang2020covid. Upper Row: COVID-19 findings; Lower Row: No COVID-19 findings
• Figure 4: Example patches for the OrgMNIST multi-class classification downstream task of the OrganSMNIST Challenges medmnistv2
• Figure 5: Example slices of the internal Brain classification downstream task. Upper Row: With brain hemorrhage; Lower Row: Without brain hemorrhage