Multi-scale Information Sharing and Selection Network with Boundary Attention for Polyp Segmentation

TL;DR

MISNet tackles polyp segmentation under challenging lighting and boundary ambiguity by introducing three core components: SSFM for selective multi-scale feature sharing, PAM for boundary-focused attention through PA-RA and PA-BA, and BWM for bottom-up refinement. Using a Res2Net backbone, MISNet first fuses multi-scale features with LFM/HFM to produce an initial guidance map via SSFM, then progressively refines boundaries with PAM and BWM under a loss combining $\mathcal{L}_{BCE}^{\omega}$ and $\mathcal{L}_{IoU}^{\omega}$ with deep supervision. Experiments on five polyp datasets show MISNet outperforms state-of-the-art methods across multiple metrics, and exhibits strong generalization to unseen datasets like ETIS, indicating robust boundary delineation and scale handling. These advances offer a practical impact by enabling more accurate and reliable polyp segmentation in diverse colonoscopy imaging scenarios.

Abstract

Polyp segmentation for colonoscopy images is of vital importance in clinical practice. It can provide valuable information for colorectal cancer diagnosis and surgery. While existing methods have achieved relatively good performance, polyp segmentation still faces the following challenges: (1) Varying lighting conditions in colonoscopy and differences in polyp locations, sizes, and morphologies. (2) The indistinct boundary between polyps and surrounding tissue. To address these challenges, we propose a Multi-scale information sharing and selection network (MISNet) for polyp segmentation task. We design a Selectively Shared Fusion Module (SSFM) to enforce information sharing and active selection between low-level and high-level features, thereby enhancing model's ability to capture comprehensive information. We then design a Parallel Attention Module (PAM) to enhance model's attention to boundaries, and a Balancing Weight Module (BWM) to facilitate the continuous refinement of boundary segmentation in the bottom-up process. Experiments on five polyp segmentation datasets demonstrate that MISNet successfully improved the accuracy and clarity of segmentation result, outperforming state-of-the-art methods.

Figures (11)

• Figure 1: Overview of the proposed method.
• Figure 2: Illustration of Low-level Fusion Module (LFM).
• Figure 3: Illustration of Selectively Shared Fusion Module (SSFM).
• Figure 4: Illustration of Parallel Attention Module (PAM).
• Figure 5: Illustration of Parallel Axial Reverse Attention (PA-RA).