Semise: Semi-supervised learning for severity representation in medical image

TL;DR

SEMISE tackles the challenge of learning severity representations in medical images with scarce labels by fusing self-supervised contrastive learning and cross-subject severity ranking. It implements a two-phase framework—Healthy-Anomaly Discrimination and Preference Optimization—integrated through a weighted loss $L_{combine} = \alpha L_{NT-Xent} + (1-\alpha) L_{PrO}$—to capture both in-context and cross-subject information. Evaluations on VinDr-Mammo, Papilledema, and ISIC demonstrate improvements in classification (F1, Recall, MAEE) and segmentation (IoU, Dice) over state-of-the-art baselines. The results suggest SEMISE provides richer severity representations that improve downstream diagnostic tasks, especially when annotated data are limited.

Abstract

This paper introduces SEMISE, a novel method for representation learning in medical imaging that combines self-supervised and supervised learning. By leveraging both labeled and augmented data, SEMISE addresses the challenge of data scarcity and enhances the encoder's ability to extract meaningful features. This integrated approach leads to more informative representations, improving performance on downstream tasks. As result, our approach achieved a 12% improvement in classification and a 3% improvement in segmentation, outperforming existing methods. These results demonstrate the potential of SIMESE to advance medical image analysis and offer more accurate solutions for healthcare applications, particularly in contexts where labeled data is limited.

Figures (3)

• Figure 1: SEMISE learning framework
• Figure 2: Heatmap of the segmentation results.
• Figure 3: Correlation between $\alpha$ and F1-Score across Three Datasets.