Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explanations of Classifiers Enhance Medical Image Segmentation via End-to-end Pre-training

Jiamin Chen, Xuhong Li, Yanwu Xu, Mengnan Du, Haoyi Xiong

TL;DR

This work tackles the data-hungry nature of medical image segmentation by introducing DoLL, a method that derives 14 diagnosis-oriented, pixel-level localization labels from multiple chest X-ray classifiers trained on CheXpert using Integrated Gradients. DoLL enables end-to-end pre-training of both backbone and segmentation modules, followed by efficient fine-tuning with segmentation adapters, achieving superior segmentation performance across lung, COVID-19 infection, and multi-organ tasks. The approach demonstrates strong improvements over baselines and offers practical advantages in training efficiency, with a publicly released CheXpert-DoLL dataset to spur further research in chest X-ray segmentation and beyond. The study also discusses limitations, robustness, and ethical considerations, outlining future avenues for generalization to other imaging modalities and broader clinical applications.

Abstract

Medical image segmentation aims to identify and locate abnormal structures in medical images, such as chest radiographs, using deep neural networks. These networks require a large number of annotated images with fine-grained masks for the regions of interest, making pre-training strategies based on classification datasets essential for sample efficiency. Based on a large-scale medical image classification dataset, our work collects explanations from well-trained classifiers to generate pseudo labels of segmentation tasks. Specifically, we offer a case study on chest radiographs and train image classifiers on the CheXpert dataset to identify 14 pathological observations in radiology. We then use Integrated Gradients (IG) method to distill and boost the explanations obtained from the classifiers, generating massive diagnosis-oriented localization labels (DoLL). These DoLL-annotated images are used for pre-training the model before fine-tuning it for downstream segmentation tasks, including COVID-19 infectious areas, lungs, heart, and clavicles. Our method outperforms other baselines, showcasing significant advantages in model performance and training efficiency across various segmentation settings.

Explanations of Classifiers Enhance Medical Image Segmentation via End-to-end Pre-training

TL;DR

This work tackles the data-hungry nature of medical image segmentation by introducing DoLL, a method that derives 14 diagnosis-oriented, pixel-level localization labels from multiple chest X-ray classifiers trained on CheXpert using Integrated Gradients. DoLL enables end-to-end pre-training of both backbone and segmentation modules, followed by efficient fine-tuning with segmentation adapters, achieving superior segmentation performance across lung, COVID-19 infection, and multi-organ tasks. The approach demonstrates strong improvements over baselines and offers practical advantages in training efficiency, with a publicly released CheXpert-DoLL dataset to spur further research in chest X-ray segmentation and beyond. The study also discusses limitations, robustness, and ethical considerations, outlining future avenues for generalization to other imaging modalities and broader clinical applications.

Abstract

Medical image segmentation aims to identify and locate abnormal structures in medical images, such as chest radiographs, using deep neural networks. These networks require a large number of annotated images with fine-grained masks for the regions of interest, making pre-training strategies based on classification datasets essential for sample efficiency. Based on a large-scale medical image classification dataset, our work collects explanations from well-trained classifiers to generate pseudo labels of segmentation tasks. Specifically, we offer a case study on chest radiographs and train image classifiers on the CheXpert dataset to identify 14 pathological observations in radiology. We then use Integrated Gradients (IG) method to distill and boost the explanations obtained from the classifiers, generating massive diagnosis-oriented localization labels (DoLL). These DoLL-annotated images are used for pre-training the model before fine-tuning it for downstream segmentation tasks, including COVID-19 infectious areas, lungs, heart, and clavicles. Our method outperforms other baselines, showcasing significant advantages in model performance and training efficiency across various segmentation settings.
Paper Structure (19 sections, 3 equations, 4 figures, 3 tables, 1 algorithm)

This paper contains 19 sections, 3 equations, 4 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methods
  4. DoLL: Diagnosis-orinted Localization Labels
  5. Boosted Distillation of Explanations
  6. Downstream Segmentation Adapters
  7. Results
  8. Experiment Settings
  9. CheXpert-DoLL Generation
  10. Segmentation Tasks
  11. Baselines and Setups
  12. Overall Results
  13. Adaptation for Downstream Tasks
  14. Efficiency
  15. Ablation Study
  16. ...and 4 more sections

Figures (4)

  • Figure 1: An Illustration of DoLL-based Pre-training Approach based on CXR Classification Datasets, Classifiers, and Explainers
  • Figure 2: Example of our DoLLs and its ground truth masks
  • Figure 3: Visualization of segmentation results on JSRT for ImageNet and DoLL pretrained models
  • Figure 4: Record of mIoU on validation set during the PSPNet fine-tuning process for JSRT, COVID-19 and Lung segmentation tasks