Weakly Supervised Patch Annotation for Improved Screening of Diabetic Retinopathy

TL;DR

Similarity-based Annotation via Feature-space Ensemble (SAFE), a two-stage framework that unifies weak supervision, contrastive learning, and patch-wise embedding inference, to systematically expand sparse annotations in the pathology, is introduced.

Abstract

Diabetic Retinopathy (DR) requires timely screening to prevent irreversible vision loss. However, its early detection remains a significant challenge since often the subtle pathological manifestations (lesions) get overlooked due to insufficient annotation. Existing literature primarily focuses on image-level supervision, weakly-supervised localization, and clustering-based representation learning, which fail to systematically annotate unlabeled lesion region(s) for refining the dataset. Expert-driven lesion annotation is labor-intensive and often incomplete, limiting the performance of deep learning models. We introduce Similarity-based Annotation via Feature-space Ensemble (SAFE), a two-stage framework that unifies weak supervision, contrastive learning, and patch-wise embedding inference, to systematically expand sparse annotations in the pathology. SAFE preserves fine-grained details of the lesion(s) under partial clinical supervision. In the first stage, a dual-arm Patch Embedding Network learns semantically structured, class-discriminative embeddings from expert annotated patches. Next, an ensemble of independent embedding spaces extrapolates labels to the unannotated regions based on spatial and semantic proximity. An abstention mechanism ensures trade-off between highly reliable annotation and noisy coverage. Experimental results demonstrate reliable separation of healthy and diseased patches, achieving upto 0.9886 accuracy. The annotation generated from SAFE substantially improves downstream tasks such as DR classification, demonstrating a substantial increase in F1-score of the diseased class and a performance gain as high as 0.545 in Area Under the Precision-Recall Curve (AUPRC). Qualitative analysis, with explainability, confirms that SAFE focuses on clinically relevant lesion patterns; and is further validated by ophthalmologists.

Figures (9)

• Figure 1: Representative patches from fundus images of Messidor dataset, illustrating key DR-related pathologies and healthy retinal regions. White bounding boxes highlight the lesions associated with DR.
• Figure 2: Overview of the SAFE framework for patch-wise label inference using weak supervision. Stage 1 takes labeled patches from weakly annotated fundus images to train multiple embedding models. Stage 2 uses the embedding spaces from Stage 1, together with unlabeled patches, to generate refined annotations for downstream tasks.
• Figure 3: Overview of the patch embedding network (PEN) architecture. The labeled input patch and its corresponding augmented version are passed through the encoder $f_\theta(\cdot)$, and a combined loss is optimized. The resulting embedding space $(\mathcal{E})$ is utilized in the second stage of SAFE.
• Figure 4: The extended confusion matrix used for computing metrics, such as Acc, BAcc, PR, RE, F1-score, AUPRC, D$_{\text{rate}}$ and MR
• Figure 5: t-SNE embeddings of labeled training (seen) and test (unseen) patches from a Messidor split, obtained using PEN optimized with (a) $\mathcal{L}_{\text{BCE}}$, (b) $\mathcal{L}_{\text{SCL}}$, (c) $\mathcal{L}$.