A multi-center analysis of deep learning methods for video polyp detection and segmentation

TL;DR

This study evaluates the applicability of deep learning techniques developed in real-time clinical colonoscopy tasks using sequence data, highlighting the critical role of temporal relationships between frames in improving diagnostic precision.

Abstract

Colonic polyps are well-recognized precursors to colorectal cancer (CRC), typically detected during colonoscopy. However, the variability in appearance, location, and size of these polyps complicates their detection and removal, leading to challenges in effective surveillance, intervention, and subsequently CRC prevention. The processes of colonoscopy surveillance and polyp removal are highly reliant on the expertise of gastroenterologists and occur within the complexities of the colonic structure. As a result, there is a high rate of missed detections and incomplete removal of colonic polyps, which can adversely impact patient outcomes. Recently, automated methods that use machine learning have been developed to enhance polyps detection and segmentation, thus helping clinical processes and reducing missed rates. These advancements highlight the potential for improving diagnostic accuracy in real-time applications, which ultimately facilitates more effective patient management. Furthermore, integrating sequence data and temporal information could significantly enhance the precision of these methods by capturing the dynamic nature of polyp growth and the changes that occur over time. To rigorously investigate these challenges, data scientists and experts gastroenterologists collaborated to compile a comprehensive dataset that spans multiple centers and diverse populations. This initiative aims to underscore the critical importance of incorporating sequence data and temporal information in the development of robust automated detection and segmentation methods. This study evaluates the applicability of deep learning techniques developed in real-time clinical colonoscopy tasks using sequence data, highlighting the critical role of temporal relationships between frames in improving diagnostic precision.

Figures (5)

• Figure 1: Sequence data samples from a range of patients demonstrate both clear frames and those with artifacts. This highlights the significance of temporal context in accurately interpreting sequence data.
• Figure 2: (top) A $t$-SNE plot for the distribution of training and testing set from the six different centers. Each point represents a frame from the multi-center dataset, color-coded according to the center originated from. The distribution highlight the challenges of obtaining a cross-center generalization and the significance of model robustness on various data from different resources. (b) Illustrates the dataset variability in terms of the number of frames from each centre and the number of polyps in these sequences.
• Figure 3: Different architectures proposed by various EndoCV2022 participating teams. Each method takes an input image, and the output prediction is generated directly or through an ensemble of networks. Table 1 presents the description of the backbone and features of various networks. Each output prediction for the detection process consists of a bounding box prediction with the class label "polyp", whereas for segmentation, it involves pixel-wise classification, with polyp pixels assigned label 1 and background pixels assigned label 0.
• Figure 4: Qualitative results for the detection task displaying both the ground truth and the outcomes from the participating teams. The first three rows illustrate variations in polyp size and location, while the fourth row features a negative sample with no polyp present.
• Figure 5: Qualitative results for the segmentation task displaying both the ground truth and the outcomes from the participating teams. The first three rows illustrate variations in polyp size and location, while the fourth row features a negative sample with no polyp present, i.e., presence of artefact.