Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transfer Risk Map: Mitigating Pixel-level Negative Transfer in Medical Segmentation

Shutong Duan, Jingyun Yang, Yang Tan, Guoqing Zhang, Yang Li, Xiao-Ping Zhang

TL;DR

The work tackles negative transfer in medical segmentation by introducing a pixel-level transferability map derived from a segmentation-adapted LEEP score. This transfer risk map guides a weighted fine-tuning loss that concentrates learning on high-transfer-hardness regions while averaging only over foreground pixels to counteract class imbalance. Empirical results on FeTS 2021 and iSeg-2019 show notable Dice gains across cross-modality and cross-task transfers and robustness in few-shot scenarios, indicating effective mitigation of negative transfer and improved generalization. The approach is simple, model-agnostic, and amenable to integration with advanced architectures and transfer-learning schemes.

Abstract

How to mitigate negative transfer in transfer learning is a long-standing and challenging issue, especially in the application of medical image segmentation. Existing methods for reducing negative transfer focus on classification or regression tasks, ignoring the non-uniform negative transfer risk in different image regions. In this work, we propose a simple yet effective weighted fine-tuning method that directs the model's attention towards regions with significant transfer risk for medical semantic segmentation. Specifically, we compute a transferability-guided transfer risk map to quantify the transfer hardness for each pixel and the potential risks of negative transfer. During the fine-tuning phase, we introduce a map-weighted loss function, normalized with image foreground size to counter class imbalance. Extensive experiments on brain segmentation datasets show our method significantly improves the target task performance, with gains of 4.37% on FeTS2021 and 1.81% on iSeg2019, avoiding negative transfer across modalities and tasks. Meanwhile, a 2.9% gain under a few-shot scenario validates the robustness of our approach.

Transfer Risk Map: Mitigating Pixel-level Negative Transfer in Medical Segmentation

TL;DR

The work tackles negative transfer in medical segmentation by introducing a pixel-level transferability map derived from a segmentation-adapted LEEP score. This transfer risk map guides a weighted fine-tuning loss that concentrates learning on high-transfer-hardness regions while averaging only over foreground pixels to counteract class imbalance. Empirical results on FeTS 2021 and iSeg-2019 show notable Dice gains across cross-modality and cross-task transfers and robustness in few-shot scenarios, indicating effective mitigation of negative transfer and improved generalization. The approach is simple, model-agnostic, and amenable to integration with advanced architectures and transfer-learning schemes.

Abstract

How to mitigate negative transfer in transfer learning is a long-standing and challenging issue, especially in the application of medical image segmentation. Existing methods for reducing negative transfer focus on classification or regression tasks, ignoring the non-uniform negative transfer risk in different image regions. In this work, we propose a simple yet effective weighted fine-tuning method that directs the model's attention towards regions with significant transfer risk for medical semantic segmentation. Specifically, we compute a transferability-guided transfer risk map to quantify the transfer hardness for each pixel and the potential risks of negative transfer. During the fine-tuning phase, we introduce a map-weighted loss function, normalized with image foreground size to counter class imbalance. Extensive experiments on brain segmentation datasets show our method significantly improves the target task performance, with gains of 4.37% on FeTS2021 and 1.81% on iSeg2019, avoiding negative transfer across modalities and tasks. Meanwhile, a 2.9% gain under a few-shot scenario validates the robustness of our approach.

Paper Structure

This paper contains 12 sections, 7 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Problem Definition
  4. Transferability Estimation on Segmentation Task
  5. Transferability Guided Transfer Risk Weighted Fine-tuning
  6. Experiments and Results
  7. Datasets
  8. Training Details
  9. Evaluation on Medical Image Segmentation Task
  10. Study on Few- shot Scenarios
  11. Effectiveness of Negative Transfer Mitigation
  12. Conclusion

Figures (2)

  • Figure 1: Illustration of the proposed transfer risk map guided weighted fine-tuning framework. The strategy consists of: 1) a transferability-based pixel-level transfer risk map, which quantifies the potential risks of negative transfer for each pixel, 2) a transfer risk map weighted fine-tuning process, which allows the model's attention towards regions with significant transfer hardness.
  • Figure 2: The pixel-level transfer risk maps before and after applying our fine-tuning method between the source model and the target data. The first row shows the results on iSeg-2019 with CSF-T1 as the source task, while the second row displays results on FeTS 2021 with 14-T1 as the source task.