Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Brain Hematoma Marker Recognition Using Multitask Learning: SwinTransformer and Swin-Unet

Kodai Hirata, Tsuyoshi Okita

TL;DR

This work tackles spurious correlations in brain CT hematoma hypodensity detection by introducing MTL-Swin-Unet, a transformer-based multitask framework that jointly trains segmentation, reconstruction, and classification with a shared Swin-Unet encoder. The segmentation and reconstruction tasks refine representations that improve downstream classification, with segmentation-based features yielding stronger AUC gains and the full three-task setup delivering peak F1 under non-covariate shift and the best AUC under covariate shift. Two learning paradigms are explored: MTL-Swin-Unet, which jointly learns all tasks, and Joint-SwinTransformer, which recycles a segmentation-trained encoder for classification via feature concatenation. Experiments on multicenter brain CT data demonstrate improved performance and explainability via Grad-CAM, highlighting practical potential for adversarially robust hematoma marker recognition in clinical settings.

Abstract

This paper proposes a method MTL-Swin-Unet which is multi-task learning using transformers for classification and semantic segmentation. For spurious-correlation problems, this method allows us to enhance the image representation with two other image representations: representation obtained by semantic segmentation and representation obtained by image reconstruction. In our experiments, the proposed method outperformed in F-value measure than other classifiers when the test data included slices from the same patient (no covariate shift). Similarly, when the test data did not include slices from the same patient (covariate shift setting), the proposed method outperformed in AUC measure.

Brain Hematoma Marker Recognition Using Multitask Learning: SwinTransformer and Swin-Unet

TL;DR

This work tackles spurious correlations in brain CT hematoma hypodensity detection by introducing MTL-Swin-Unet, a transformer-based multitask framework that jointly trains segmentation, reconstruction, and classification with a shared Swin-Unet encoder. The segmentation and reconstruction tasks refine representations that improve downstream classification, with segmentation-based features yielding stronger AUC gains and the full three-task setup delivering peak F1 under non-covariate shift and the best AUC under covariate shift. Two learning paradigms are explored: MTL-Swin-Unet, which jointly learns all tasks, and Joint-SwinTransformer, which recycles a segmentation-trained encoder for classification via feature concatenation. Experiments on multicenter brain CT data demonstrate improved performance and explainability via Grad-CAM, highlighting practical potential for adversarially robust hematoma marker recognition in clinical settings.

Abstract

This paper proposes a method MTL-Swin-Unet which is multi-task learning using transformers for classification and semantic segmentation. For spurious-correlation problems, this method allows us to enhance the image representation with two other image representations: representation obtained by semantic segmentation and representation obtained by image reconstruction. In our experiments, the proposed method outperformed in F-value measure than other classifiers when the test data included slices from the same patient (no covariate shift). Similarly, when the test data did not include slices from the same patient (covariate shift setting), the proposed method outperformed in AUC measure.
Paper Structure (26 sections, 6 equations, 4 figures, 1 table)

This paper contains 26 sections, 6 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. Multi-task Learning with Transformer
  4. Joint Learning
  5. Method
  6. Multi-task Learning Using Swin-Unet
  7. Architecture
  8. Decoder for Segmentation and Image Reconstruction Tasks
  9. Classification Head
  10. Loss Function
  11. Joint Learning Using Swin-Unet
  12. Architecture
  13. Encoder with Representation for Segmentation Task
  14. Encoder during Class Classification
  15. Classification Head
  16. ...and 11 more sections

Figures (4)

  • Figure 1: Architecture of MTL-Swin-Unet
  • Figure 2: Architecture of Joint-SwinTransformer
  • Figure 3: (a) Hypodensity (b) Images without hypodensity overlapping with other hemorrhage classes (from left to right: image without hemorrhage, margin irregular sign, blend sign, fluid level)
  • Figure 4: The visualization results of Grad-CAM using MTL-Swin-Unet (cls + seg + rec) for (a) hypodensity images and (b) images with hemorrhage but not hypodensity