Adaptive Bidirectional Displacement for Semi-Supervised Medical Image Segmentation

TL;DR

This work designs a bidirectional patch displacement based on reliable prediction confidence for unlabeled data to generate new samples, which can effectively suppress uncontrollable regions and still retain the influence of input perturbations.

Abstract

Consistency learning is a central strategy to tackle unlabeled data in semi-supervised medical image segmentation (SSMIS), which enforces the model to produce consistent predictions under the perturbation. However, most current approaches solely focus on utilizing a specific single perturbation, which can only cope with limited cases, while employing multiple perturbations simultaneously is hard to guarantee the quality of consistency learning. In this paper, we propose an Adaptive Bidirectional Displacement (ABD) approach to solve the above challenge. Specifically, we first design a bidirectional patch displacement based on reliable prediction confidence for unlabeled data to generate new samples, which can effectively suppress uncontrollable regions and still retain the influence of input perturbations. Meanwhile, to enforce the model to learn the potentially uncontrollable content, a bidirectional displacement operation with inverse confidence is proposed for the labeled images, which generates samples with more unreliable information to facilitate model learning. Extensive experiments show that ABD achieves new state-of-the-art performances for SSMIS, significantly improving different baselines. Source code is available at https://github.com/chy-upc/ABD.

Figures (3)

• Figure 1: Illustration of prediction results. (a) Using only network perturbation; (b) Combining network perturbation with input perturbation; (c) Incorporating ABD on top of the two perturbations. Using a single perturbation has limitations while using multiple perturbations makes it uncontrollable. Introducing ABD greatly alleviates the issue and allows the model to perform significantly better. The white dashed boxes highlight the regions with wrong predictions.
• Figure 2: Overview of our adaptive bidirectional displacement framework. (A) For the unlabeled data, one image is subjected to weak and strong augmentations, resulting in two images that are separately input to two networks for cross-supervision. Then, based on the $A_w^u$, $Z_w^u$, $A_s^u$, and $Z_s^u$, the patches in the images are bidirectionally displaced, resulting in the formation of new samples ${X}_{s \rightarrow w}^u$ and ${X}_{w \rightarrow s}^u$. These new samples are further fed into the networks for cross-supervision. (B) For the labeled images, they are also subjected to both weak and strong augmentations, and their predictions are supervised by the labels. Afterward, based on the $A_w^l$ and $A_s^l$, inverse bidirectional patch displacement is performed on the images, resulting in the generation of new samples ${X}_{s \rightarrow w}^l$ and ${X}_{w \rightarrow s}^l$. Similarly, the labels undergo the same operation, leading to the creation of new labels $Y_{s\rightarrow w}^l$ and $Y_{w\rightarrow s}^l$. The new samples are then fed into the network, and their predictions are supervised by the new labels. Note that ABD-R and ABD-I are two parallel modules during training.
• Figure 3: Visualization of segmentation results on ACDC dataset with 10% labeled data. (a) Ground-truth. (b) Cross Teaching results. (c) Ours-ABD (Cross Teaching) results. (d) BCP results. (e) Ours-ABD (BCP) results. Best viewed in color on the screen.