SM2C: Boost the Semi-supervised Segmentation for Medical Image by using Meta Pseudo Labels and Mixed Images

TL;DR

This work introduces a novel method called Scaling-up Mix with Multi-Class (SM2C), which uses three strategies - scaling-up image size, multi-class mixing, and object shape jittering - to improve the ability to learn semantic features within medical images.

Abstract

Recently, machine learning-based semantic segmentation algorithms have demonstrated their potential to accurately segment regions and contours in medical images, allowing the precise location of anatomical structures and abnormalities. Although medical images are difficult to acquire and annotate, semi-supervised learning methods are efficient in dealing with the scarcity of labeled data. However, overfitting is almost inevitable due to the limited images for training. Furthermore, the intricate shapes of organs and lesions in medical images introduce additional complexity in different cases, preventing networks from acquiring a strong ability to generalize. To this end, we introduce a novel method called Scaling-up Mix with Multi-Class (SM2C). This method uses three strategies - scaling-up image size, multi-class mixing, and object shape jittering - to improve the ability to learn semantic features within medical images. By diversifying the shape of the segmentation objects and enriching the semantic information within each sample, the SM2C demonstrates its potential, especially in the training of unlabelled data. Extensive experiments demonstrate the effectiveness of the SM2C on three benchmark medical image segmentation datasets. The proposed framework shows significant improvements over state-of-the-art counterparts.

Figures (8)

• Figure 1: Different sections of a heart. (a) represents the basal section, which lacks regions of the right ventricle(RV), myocardium, and the left ventricle(LV); (b), (c), and (d) approach the apical section, while the RV, myocardium, and LV vary in size and shape.
• Figure 2: Illustration of Scaling-up Mix with Multi-Class (SM$^2$C). Four unlabeled images (quantity unlimited) extract their respective segmentation objects, augment these objects, randomly mix them with other images, and finally, through a concatenation operation, create a larger-sized input image containing more content.
• Figure 3: Illustration of Scaling-up Mix, an important part of SM$^2$C. Scaling-up Mix creates images with increased foreground-background diversity by concatenating four images.
• Figure 4: The Multi-Class-Jittering Mix operation first extracts the segmentation objects from each original image. After applying transformations such as deformation and translation to these objects, they are mixed into one of the original images. This process simulates the diversity of organ morphology as well as the diversity between foreground and background in medical images.
• Figure 5: Illustration of the application of SM$^2$C in a semi-supervised framework based on MPL. The framework consists of two networks: the teacher network and the student network. The teacher network generates pseudo-labels for unlabelled images and the student network trains on these samples. In each iteration, the feedback generated by the student network guides the training of the teacher network. In addition, the UDA part, based on SM$^2$C, allows the teacher network to be trained on unlabelled images, allowing the teacher network to learn additional semantic information beyond the labeled images, thereby generating more reliable pseudo-labels.