Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Attention Gated with Hybrid Dual Pyramid Transformer-CNN for Generalized Segmentation in Medical Imaging

Fares Bougourzi, Fadi Dornaika, Abdelmalik Taleb-Ahmed, Vinh Truong Hoang

TL;DR

The paper tackles generalization in medical image segmentation under data variability by proposing PAG-TransYnet, a hybrid CNN-Transformer encoder. It combines a Pyramid Encoder, a CNN-focused main path, and a Transformer stream, fused via Dual-Attention Gates to capture both local and global context. Key contributions include the Pyramid path, integration of PVT-v2 and ViT stages, and a novel fusion mechanism that yields state-of-the-art results across abdominal organ, infection, and microscopic segmentation tasks. The approach offers a robust, scalable framework for efficient CNN-Transformer fusion in medical imaging, with publicly available code to facilitate adoption and further research.

Abstract

Inspired by the success of Transformers in Computer vision, Transformers have been widely investigated for medical imaging segmentation. However, most of Transformer architecture are using the recent transformer architectures as encoder or as parallel encoder with the CNN encoder. In this paper, we introduce a novel hybrid CNN-Transformer segmentation architecture (PAG-TransYnet) designed for efficiently building a strong CNN-Transformer encoder. Our approach exploits attention gates within a Dual Pyramid hybrid encoder. The contributions of this methodology can be summarized into three key aspects: (i) the utilization of Pyramid input for highlighting the prominent features at different scales, (ii) the incorporation of a PVT transformer to capture long-range dependencies across various resolutions, and (iii) the implementation of a Dual-Attention Gate mechanism for effectively fusing prominent features from both CNN and Transformer branches. Through comprehensive evaluation across different segmentation tasks including: abdominal multi-organs segmentation, infection segmentation (Covid-19 and Bone Metastasis), microscopic tissues segmentation (Gland and Nucleus). The proposed approach demonstrates state-of-the-art performance and exhibits remarkable generalization capabilities. This research represents a significant advancement towards addressing the pressing need for efficient and adaptable segmentation solutions in medical imaging applications.

Rethinking Attention Gated with Hybrid Dual Pyramid Transformer-CNN for Generalized Segmentation in Medical Imaging

TL;DR

The paper tackles generalization in medical image segmentation under data variability by proposing PAG-TransYnet, a hybrid CNN-Transformer encoder. It combines a Pyramid Encoder, a CNN-focused main path, and a Transformer stream, fused via Dual-Attention Gates to capture both local and global context. Key contributions include the Pyramid path, integration of PVT-v2 and ViT stages, and a novel fusion mechanism that yields state-of-the-art results across abdominal organ, infection, and microscopic segmentation tasks. The approach offers a robust, scalable framework for efficient CNN-Transformer fusion in medical imaging, with publicly available code to facilitate adoption and further research.

Abstract

Inspired by the success of Transformers in Computer vision, Transformers have been widely investigated for medical imaging segmentation. However, most of Transformer architecture are using the recent transformer architectures as encoder or as parallel encoder with the CNN encoder. In this paper, we introduce a novel hybrid CNN-Transformer segmentation architecture (PAG-TransYnet) designed for efficiently building a strong CNN-Transformer encoder. Our approach exploits attention gates within a Dual Pyramid hybrid encoder. The contributions of this methodology can be summarized into three key aspects: (i) the utilization of Pyramid input for highlighting the prominent features at different scales, (ii) the incorporation of a PVT transformer to capture long-range dependencies across various resolutions, and (iii) the implementation of a Dual-Attention Gate mechanism for effectively fusing prominent features from both CNN and Transformer branches. Through comprehensive evaluation across different segmentation tasks including: abdominal multi-organs segmentation, infection segmentation (Covid-19 and Bone Metastasis), microscopic tissues segmentation (Gland and Nucleus). The proposed approach demonstrates state-of-the-art performance and exhibits remarkable generalization capabilities. This research represents a significant advancement towards addressing the pressing need for efficient and adaptable segmentation solutions in medical imaging applications.
Paper Structure (12 sections, 5 figures, 5 tables)

This paper contains 12 sections, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Proposed Approach
  4. Pyramid Encoder
  5. Main Encoder: Attention Fusion
  6. Dual-Attention Gate
  7. Datasets and tasks
  8. Experiments and Results
  9. Experimental Setup
  10. Results
  11. Ablation Study
  12. Conclusion

Figures (5)

  • Figure 1: Examples of Medical Imaging Segmentation, the first, second and third rows represent the input image, ground truth and the prediction of our approach, respectively. First, second, third, fourth and fifth columns depict abdominal multi-organ segmentation, Covid-19, Bone Metastasis, Gland, and Nucleus, respectively.
  • Figure 2: Our proposed PAG-TransYnet architecture.
  • Figure 3: The detailed description of our proposed PAG-TransYnet approach.
  • Figure 4: Detailed representation of the convolutional blocks used in our proposed PAG-TransYnet architecture.
  • Figure 5: The proposed Dual-Attention Gate, left the general structure and the right is the detailed one.