Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GPF-Net: Gated Progressive Fusion Learning for Polyp Re-Identification

Suncheng Xiang, Xiaoyang Wang, Junjie Jiang, Hejia Wang, Dahong Qian

TL;DR

The paper addresses polyp re-identification across views by introducing GPF-Net, a multimodal framework that progressively fuses visual and textual features through gated, multi-layer interactions. A dynamic gating mechanism tunes the contribution of each modality at multiple fusion stages, implemented via a Transformer-based architecture. Empirical results on Colo-Pair and standard ReID datasets show substantial improvements in mAP and Rank-1, with ablations highlighting the strong value of multimodal fusion and adaptive gating. The approach achieves competitive computational efficiency and offers a scalable pathway for multimodal medical image retrieval tasks.

Abstract

Colonoscopic Polyp Re-Identification aims to match the same polyp from a large gallery with images from different views taken using different cameras, which plays an important role in the prevention and treatment of colorectal cancer in computer-aided diagnosis. However, the coarse resolution of high-level features of a specific polyp often leads to inferior results for small objects where detailed information is important. To address this challenge, we propose a novel architecture, named Gated Progressive Fusion network, to selectively fuse features from multiple levels using gates in a fully connected way for polyp ReID. On the basis of it, a gated progressive fusion strategy is introduced to achieve layer-wise refinement of semantic information through multi-level feature interactions. Experiments on standard benchmarks show the benefits of the multimodal setting over state-of-the-art unimodal ReID models, especially when combined with the specialized multimodal fusion strategy.

GPF-Net: Gated Progressive Fusion Learning for Polyp Re-Identification

TL;DR

The paper addresses polyp re-identification across views by introducing GPF-Net, a multimodal framework that progressively fuses visual and textual features through gated, multi-layer interactions. A dynamic gating mechanism tunes the contribution of each modality at multiple fusion stages, implemented via a Transformer-based architecture. Empirical results on Colo-Pair and standard ReID datasets show substantial improvements in mAP and Rank-1, with ablations highlighting the strong value of multimodal fusion and adaptive gating. The approach achieves competitive computational efficiency and offers a scalable pathway for multimodal medical image retrieval tasks.

Abstract

Colonoscopic Polyp Re-Identification aims to match the same polyp from a large gallery with images from different views taken using different cameras, which plays an important role in the prevention and treatment of colorectal cancer in computer-aided diagnosis. However, the coarse resolution of high-level features of a specific polyp often leads to inferior results for small objects where detailed information is important. To address this challenge, we propose a novel architecture, named Gated Progressive Fusion network, to selectively fuse features from multiple levels using gates in a fully connected way for polyp ReID. On the basis of it, a gated progressive fusion strategy is introduced to achieve layer-wise refinement of semantic information through multi-level feature interactions. Experiments on standard benchmarks show the benefits of the multimodal setting over state-of-the-art unimodal ReID models, especially when combined with the specialized multimodal fusion strategy.
Paper Structure (12 sections, 3 equations, 2 figures, 4 tables)

This paper contains 12 sections, 3 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Preliminary
  4. Our Proposed GPF-Net Framework
  5. Dynamic Gating Progressive Fusion Mechanism
  6. Experiments
  7. Datasets and Evaluation Metric
  8. Implementation details
  9. Comparison with State-of-the-Arts
  10. Ablation Studies
  11. Parameter Analysis
  12. Conclusion

Figures (2)

  • Figure 1: The overall architecture of our proposed GPF-Net method, which consists of gated progressive fusion net and self-attention layers.
  • Figure 2: Visualization of ranking results (left) and attention maps(right): (a) Original images; (b) CNN-based training method without text information; (c) Our gated progressive fusion strategy.