Multi-source adversarial transfer learning for ultrasound image segmentation with limited similarity

TL;DR

This work addresses ultrasound image segmentation with scarce labeled data and limited similarity between potential source domains. It introduces a multi-source adversarial transfer learning framework that uses separate source-domain sub-networks, a gradient reversal domain classifier, and a fusion-based target predictor, coupled with a multi-domain independence strategy to balance batches and maximize unlabeled target data use. The approach yields superior segmentation performance across BUSI, DDTI, and CT2USforKidneySeg datasets, with ablations confirming the value of multiple sources and the independence mechanism, particularly as labeled data diminish. The method holds practical significance for deploying robust ultrasound segmentation models in data-scarce clinical settings, enabling better generalization across organs and scanners without extensive labeling.

Abstract

Lesion segmentation of ultrasound medical images based on deep learning techniques is a widely used method for diagnosing diseases. Although there is a large amount of ultrasound image data in medical centers and other places, labeled ultrasound datasets are a scarce resource, and it is likely that no datasets are available for new tissues/organs. Transfer learning provides the possibility to solve this problem, but there are too many features in natural images that are not related to the target domain. As a source domain, redundant features that are not conducive to the task will be extracted. Migration between ultrasound images can avoid this problem, but there are few types of public datasets, and it is difficult to find sufficiently similar source domains. Compared with natural images, ultrasound images have less information, and there are fewer transferable features between different ultrasound images, which may cause negative transfer. To this end, a multi-source adversarial transfer learning network for ultrasound image segmentation is proposed. Specifically, to address the lack of annotations, the idea of adversarial transfer learning is used to adaptively extract common features between a certain pair of source and target domains, which provides the possibility to utilize unlabeled ultrasound data. To alleviate the lack of knowledge in a single source domain, multi-source transfer learning is adopted to fuse knowledge from multiple source domains. In order to ensure the effectiveness of the fusion and maximize the use of precious data, a multi-source domain independent strategy is also proposed to improve the estimation of the target domain data distribution, which further increases the learning ability of the multi-source adversarial migration learning network in multiple domains.

Figures (13)

• Figure 1: A multi-source domain has the potential to provide more adequate generic features than a single source domain.
• Figure 2: Multiple Sub-networks use the feature fusion layer to form the Structure multi-source adversarial transfer learning network.
• Figure 3: U-Net has the structure of the Down-sampling Encoding, Up-sampling Decoding and Jumping Connection, which combines a more detailed graphic and deep features that help target classification.
• Figure 4: The $i\hbox{-}th$ Sub-network consists of the $i\hbox{-}th$ Feature extractor $G_e^i\left( \cdot ; \theta_e^i \right)$, the $i\hbox{-}th$ Domain Classifier $G_c^i\left( \cdot ; \theta_c^i \right)$, the $i\hbox{-}th$ Source segmentation predictor $G_s^i\left( \cdot ; \theta_s^i \right)$, and the Target segmentation predictor $G_t\left( \cdot; \theta_t \right)$
• Figure 5: Generate multiple Sub-batch data with the Multi-domain independence strategy and then constitute the batch data.