TSdetector: Temporal-Spatial Self-correction Collaborative Learning for Colonoscopy Video Detection

TL;DR

A novel Temporal-Spatial self-correction detector (TSdetector), which first integrates temporal-level consistency learning and spatial-level reliability learning to detect objects continuously and achieves the highest polyp detection rate and outperforms other state-of-the-art methods.

Abstract

CNN-based object detection models that strike a balance between performance and speed have been gradually used in polyp detection tasks. Nevertheless, accurately locating polyps within complex colonoscopy video scenes remains challenging since existing methods ignore two key issues: intra-sequence distribution heterogeneity and precision-confidence discrepancy. To address these challenges, we propose a novel Temporal-Spatial self-correction detector (TSdetector), which first integrates temporal-level consistency learning and spatial-level reliability learning to detect objects continuously. Technically, we first propose a global temporal-aware convolution, assembling the preceding information to dynamically guide the current convolution kernel to focus on global features between sequences. In addition, we designed a hierarchical queue integration mechanism to combine multi-temporal features through a progressive accumulation manner, fully leveraging contextual consistency information together with retaining long-sequence-dependency features. Meanwhile, at the spatial level, we advance a position-aware clustering to explore the spatial relationships among candidate boxes for recalibrating prediction confidence adaptively, thus eliminating redundant bounding boxes efficiently. The experimental results on three publicly available polyp video dataset show that TSdetector achieves the highest polyp detection rate and outperforms other state-of-the-art methods. The code can be available at https://github.com/soleilssss/TSdetector.

Figures (16)

• Figure 1: Two key challenges in video polyp detection: intra-sequence distribution heterogeneity and precision-confidence discrepancy.
• Figure 2: Comparison between classical detection model paradigms a) and our temporal-spatial self-correcting detector b). In contrast, our TSdetector utilizes spatial and temporal information to compensate for the limitations of traditional detection models from three perspectives.
• Figure 3: Comparison between the limitations of existing detection frameworks and the advantages of the proposed method. a) and b) represent two solution ideas for the challenge: temporal-level consistency learning and spatial-level reliability learning.
• Figure 4: The overview of temporal-spatial detector architecture consists of temporal-level consistency learning and spatial-level reliability learning. a) & b) At the temporal level, we aim to enhance the flexibility of feature extraction and fusion, thereby generating more reliable proposals. c) At the spatial level, we aim to reduce discrepancies between the confidence scores and the actual positive probabilities of candidate bounding boxes.
• Figure 5: Global temporal-aware convolution differs from conventional convolutions in that its parameters can be adaptively adjusted in each frame. The temporal calibration factor is generated from the feature sequence of previous frames.