Using Multi-Instance Learning to Identify Unique Polyps in Colon Capsule Endoscopy Images

TL;DR

Problem: identify unique polyps in colon capsule endoscopy (CCE) images despite labeling ambiguity. Approach: formulate as multi-instance verification (MIV) within a Siamese framework, leveraging attention mechanisms (variance-excited multi-head attention and distance-based attention) and self-supervised SimCLR pretraining to produce robust embeddings. Key contributions: extensive evaluation across backbones, attention variants, and SimCLR pretraining showing substantial gains (e.g., DBA-L1 with 2 heads on ConvNeXt with SimCLR achieves 86.26% test accuracy and 0.928 AUC); analysis includes misclassification patterns and practical dataset considerations. Significance: enables scalable, automated analysis of large CCE image streams, reduces clinician workload, and provides a framework adaptable to other medical imaging tasks with weakly labeled image bags.

Abstract

Identifying unique polyps in colon capsule endoscopy (CCE) images is a critical yet challenging task for medical personnel due to the large volume of images, the cognitive load it creates for clinicians, and the ambiguity in labeling specific frames. This paper formulates this problem as a multi-instance learning (MIL) task, where a query polyp image is compared with a target bag of images to determine uniqueness. We employ a multi-instance verification (MIV) framework that incorporates attention mechanisms, such as variance-excited multi-head attention (VEMA) and distance-based attention (DBA), to enhance the model's ability to extract meaningful representations. Additionally, we investigate the impact of self-supervised learning using SimCLR to generate robust embeddings. Experimental results on a dataset of 1912 polyps from 754 patients demonstrate that attention mechanisms significantly improve performance, with DBA L1 achieving the highest test accuracy of 86.26\% and a test AUC of 0.928 using a ConvNeXt backbone with SimCLR pretraining. This study underscores the potential of MIL and self-supervised learning in advancing automated analysis of Colon Capsule Endoscopy images, with implications for broader medical imaging applications.

Figures (6)

• Figure 1: Positive exemplar images with each row showing five images of a unique polyp, where in each row, the first image corresponds to first partial and fifth image is the last full view of the same polyp. The polyps have been marked in green for each image.
• Figure 2: A diagram showing the framework used for MIV in this study. The feature extraction is performed for both the query image and the target bag. Multi-head attention mechanisms are used to extract representations from the query, denoted by $V_Q$, and the target set, denoted by $V_T$. These representations are then used by a Siamese network for the classification task.
• Figure 3: True Positives (Pred = true, Label = true), False Negatives (Pred = false, Label = true), False Positives (Pred = true, Label = false), True Negatives (Pred = false, Label = false) for the DBA L2(h=2) model using the pretrained ConvNeXt. In each row, the leftmost image is the query and the 4 images to the right of each query are the target images.
• Figure 4: True Positives (Pred = true, Label = true), False Negatives (Pred = false, Label = true), False Positives (Pred = true, Label = false),True Negatives (Pred = false, Label = false) for the DBA L1(h=2) model applying SimCLR using the ConvNeXt backbone. In each row, the leftmost image is the query and the 4 images to the right of each query are the target images.
• Figure 5: Confusion matrices from the best MIV model associated with (left) the pretrained ConvNext, and (right) SimCLR pretraining using the ConvNext backbone