Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CMSA-Net: Causal Multi-scale Aggregation with Adaptive Multi-source Reference for Video Polyp Segmentation

Tong Wang, Yaolei Qi, Siwen Wang, Imran Razzak, Guanyu Yang, Yutong Xie

TL;DR

This work proposes a robust VPS framework named CMSA-Net, which introduces a Causal Multi-scale Aggregation module to effectively gather semantic information from multiple historical frames at different scales, and designs a Dynamic Multi-source Reference strategy that adaptively selects informative and reliable reference frames based on semantic separability and prediction confidence.

Abstract

Video polyp segmentation (VPS) is an important task in computer-aided colonoscopy, as it helps doctors accurately locate and track polyps during examinations. However, VPS remains challenging because polyps often look similar to surrounding mucosa, leading to weak semantic discrimination. In addition, large changes in polyp position and scale across video frames make stable and accurate segmentation difficult. To address these challenges, we propose a robust VPS framework named CMSA-Net. The proposed network introduces a Causal Multi-scale Aggregation (CMA) module to effectively gather semantic information from multiple historical frames at different scales. By using causal attention, CMA ensures that temporal feature propagation follows strict time order, which helps reduce noise and improve feature reliability. Furthermore, we design a Dynamic Multi-source Reference (DMR) strategy that adaptively selects informative and reliable reference frames based on semantic separability and prediction confidence. This strategy provides strong multi-frame guidance while keeping the model efficient for real-time inference. Extensive experiments on the SUN-SEG dataset demonstrate that CMSA-Net achieves state-of-the-art performance, offering a favorable balance between segmentation accuracy and real-time clinical applicability.

CMSA-Net: Causal Multi-scale Aggregation with Adaptive Multi-source Reference for Video Polyp Segmentation

TL;DR

This work proposes a robust VPS framework named CMSA-Net, which introduces a Causal Multi-scale Aggregation module to effectively gather semantic information from multiple historical frames at different scales, and designs a Dynamic Multi-source Reference strategy that adaptively selects informative and reliable reference frames based on semantic separability and prediction confidence.

Abstract

Video polyp segmentation (VPS) is an important task in computer-aided colonoscopy, as it helps doctors accurately locate and track polyps during examinations. However, VPS remains challenging because polyps often look similar to surrounding mucosa, leading to weak semantic discrimination. In addition, large changes in polyp position and scale across video frames make stable and accurate segmentation difficult. To address these challenges, we propose a robust VPS framework named CMSA-Net. The proposed network introduces a Causal Multi-scale Aggregation (CMA) module to effectively gather semantic information from multiple historical frames at different scales. By using causal attention, CMA ensures that temporal feature propagation follows strict time order, which helps reduce noise and improve feature reliability. Furthermore, we design a Dynamic Multi-source Reference (DMR) strategy that adaptively selects informative and reliable reference frames based on semantic separability and prediction confidence. This strategy provides strong multi-frame guidance while keeping the model efficient for real-time inference. Extensive experiments on the SUN-SEG dataset demonstrate that CMSA-Net achieves state-of-the-art performance, offering a favorable balance between segmentation accuracy and real-time clinical applicability.
Paper Structure (11 sections, 2 equations, 5 figures, 4 tables)

This paper contains 11 sections, 2 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Causal Multi-scale Aggregation (CMA) Module
  4. Dynamic Multi-source Reference (DMR) Strategy
  5. Learning Process and Implementation Details
  6. Experiments
  7. Dataset and Metrics
  8. Comparisons with State-of-the-art Methods
  9. Ablation Study
  10. Conclusion
  11. More Qualitative Results

Figures (5)

  • Figure 1: (a) Challenge 1: Weak semantic discrimination, caused by the low contrast between polyps and background (case14_6 and case19). (b) Challenge 2: Large spatio-temporal variations across frames (case51_1 and case71_1). (c) Limitation 1: Existing feature fusion methods for polyp semantic learning , which ignore multi-scale information and intrinsic identity relationships. (d) Limitation 2: Current approaches adopt a fixed single-source reference, failing to capture dynamic and diverse spatio-temporal cues. (e) Our Method.
  • Figure 2: Framework of the proposed CMSA-Net, including the CMA module (\ref{['subsec:cma']}) with DMR strategy (\ref{['subsec:dmr']}).
  • Figure 3: Qualitative Comparisons. Upper (case91_2): a sequence of consecutive low-contrast frames; Lower (case32_4): significant variations between adjacent frames.
  • Figure 4: Qualitative Comparisons on consecutive low-contrast frames (case91_2 and case14_6).
  • Figure 5: Qualitative Comparisons on consecutive low-contrast and significant-variation frames (case51_1 and case74_1).