Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Perspective+ Unet: Enhancing Segmentation with Bi-Path Fusion and Efficient Non-Local Attention for Superior Receptive Fields

Jintong Hu, Siyan Chen, Zhiyi Pan, Sen Zeng, Wenming Yang

TL;DR

The paper tackles the challenge of capturing long-range spatial dependencies in 3D medical image segmentation, where standard CNNs have limited local receptive fields and global attention methods incur high computational cost. It introduces Perspective+ Unet, featuring a Bi-Path encoder to fuse local and dilated context, an Efficient Non-Local Transformer Block (ENLTB) with linear-complexity ENLSA for global modeling, and a Spatial Cross-Scale Integrator (SCSI) to merge multi-scale information. The approach yields improved receptive fields and segmentation quality, demonstrated on Synapse and ACDC with ablations validating the contributions of BPRB, ENLTB, and SCSI, achieving DSC around 84.63% and HD around 11.74 on Synapse. The work offers a practical, efficient framework for accurate 3D medical segmentation with clear potential for clinical deployment, and the code is provided in the supplementary material.

Abstract

Precise segmentation of medical images is fundamental for extracting critical clinical information, which plays a pivotal role in enhancing the accuracy of diagnoses, formulating effective treatment plans, and improving patient outcomes. Although Convolutional Neural Networks (CNNs) and non-local attention methods have achieved notable success in medical image segmentation, they either struggle to capture long-range spatial dependencies due to their reliance on local features, or face significant computational and feature integration challenges when attempting to address this issue with global attention mechanisms. To overcome existing limitations in medical image segmentation, we propose a novel architecture, Perspective+ Unet. This framework is characterized by three major innovations: (i) It introduces a dual-pathway strategy at the encoder stage that combines the outcomes of traditional and dilated convolutions. This not only maintains the local receptive field but also significantly expands it, enabling better comprehension of the global structure of images while retaining detail sensitivity. (ii) The framework incorporates an efficient non-local transformer block, named ENLTB, which utilizes kernel function approximation for effective long-range dependency capture with linear computational and spatial complexity. (iii) A Spatial Cross-Scale Integrator strategy is employed to merge global dependencies and local contextual cues across model stages, meticulously refining features from various levels to harmonize global and local information. Experimental results on the ACDC and Synapse datasets demonstrate the effectiveness of our proposed Perspective+ Unet. The code is available in the supplementary material.

Perspective+ Unet: Enhancing Segmentation with Bi-Path Fusion and Efficient Non-Local Attention for Superior Receptive Fields

TL;DR

The paper tackles the challenge of capturing long-range spatial dependencies in 3D medical image segmentation, where standard CNNs have limited local receptive fields and global attention methods incur high computational cost. It introduces Perspective+ Unet, featuring a Bi-Path encoder to fuse local and dilated context, an Efficient Non-Local Transformer Block (ENLTB) with linear-complexity ENLSA for global modeling, and a Spatial Cross-Scale Integrator (SCSI) to merge multi-scale information. The approach yields improved receptive fields and segmentation quality, demonstrated on Synapse and ACDC with ablations validating the contributions of BPRB, ENLTB, and SCSI, achieving DSC around 84.63% and HD around 11.74 on Synapse. The work offers a practical, efficient framework for accurate 3D medical segmentation with clear potential for clinical deployment, and the code is provided in the supplementary material.

Abstract

Precise segmentation of medical images is fundamental for extracting critical clinical information, which plays a pivotal role in enhancing the accuracy of diagnoses, formulating effective treatment plans, and improving patient outcomes. Although Convolutional Neural Networks (CNNs) and non-local attention methods have achieved notable success in medical image segmentation, they either struggle to capture long-range spatial dependencies due to their reliance on local features, or face significant computational and feature integration challenges when attempting to address this issue with global attention mechanisms. To overcome existing limitations in medical image segmentation, we propose a novel architecture, Perspective+ Unet. This framework is characterized by three major innovations: (i) It introduces a dual-pathway strategy at the encoder stage that combines the outcomes of traditional and dilated convolutions. This not only maintains the local receptive field but also significantly expands it, enabling better comprehension of the global structure of images while retaining detail sensitivity. (ii) The framework incorporates an efficient non-local transformer block, named ENLTB, which utilizes kernel function approximation for effective long-range dependency capture with linear computational and spatial complexity. (iii) A Spatial Cross-Scale Integrator strategy is employed to merge global dependencies and local contextual cues across model stages, meticulously refining features from various levels to harmonize global and local information. Experimental results on the ACDC and Synapse datasets demonstrate the effectiveness of our proposed Perspective+ Unet. The code is available in the supplementary material.
Paper Structure (14 sections, 8 equations, 5 figures, 7 tables)

This paper contains 14 sections, 8 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Bi-Path Residual Block (BPRB)
  4. Efficient Non-Local Transformer Block (ENLTB)
  5. Spatial Cross-Scale Integrator (SCSI)
  6. Experiments
  7. Datasets and Evaluation
  8. Synapse:
  9. ACDC:
  10. Experiment Details
  11. Quantitative and qualitative segmentation results
  12. Ablation Study
  13. Conclusion
  14. Supplementary Material

Figures (5)

  • Figure 1: The pipeline of the proposed Perspective$+$ Unet. The model consists of (i) a bi-path CNN based encoder to effectively capture local details and broad contextual information, (ii) a bottleneck composed of Efficient Non-Local Transformer Blocks and Spatial Cross-Scale Integrator for enhanced global perspective, and (iii) a decoder to incorporate both global and local information for generating segmentation results.
  • Figure 2: Visualized segmentation results of different methods on the Synapse multi-organ CT dataset. Our method (the last column) exhibits the smoothest boundaries and the most accurate segmentation outcomes.
  • Figure 3: Visualization of attention heat maps from the intermediate layers of the network. Highlighting areas are closely aligned with segmentation labels, demonstrating our Perspective$+$ Unet's accuracy in feature identification and localization.
  • Figure 4: Visualized segmentation results of different methods on the Synapse multi-organ CT dataset. Our method (the last column) exhibits the smoothest boundaries and the most accurate segmentation outcomes.
  • Figure 5: Visualization of attention heat maps from the intermediate layers of the network. Highlighting areas are closely aligned with segmentation labels, demonstrating our Perspective$+$ Unet's accuracy in feature identification and localization.