ColonMapper: topological mapping and localization for colonoscopy

TL;DR

ColonMapper tackles the problem of mapping and localizing within real colonoscopies by adopting a topological SLAM approach. It builds a graph where nodes are sets of covisible images representing colon places and uses LoFTR for short-term covisibility and a deep global descriptor for long-term place recognition, supplemented by a Bayesian filter for robust relocalization. The key contributions include a novel node-based topological mapping strategy, a deep global descriptor trained on real colonoscopy data with hard-positive mining, and a Bayesian localization framework that can reject spurious observations to improve reliability. Empirical results on phantom and real colonoscopy datasets demonstrate accurate intra- and cross-sequence localization, suggesting ColonMapper as a practical step toward deep colonoscopic SLAM and more robust navigation within the colon.

Abstract

We propose a topological mapping and localization system able to operate on real human colonoscopies, despite significant shape and illumination changes. The map is a graph where each node codes a colon location by a set of real images, while edges represent traversability between nodes. For close-in-time images, where scene changes are minor, place recognition can be successfully managed with the recent transformers-based local feature matching algorithms. However, under long-term changes -- such as different colonoscopies of the same patient -- feature-based matching fails. To address this, we train on real colonoscopies a deep global descriptor achieving high recall with significant changes in the scene. The addition of a Bayesian filter boosts the accuracy of long-term place recognition, enabling relocalization in a previously built map. Our experiments show that ColonMapper is able to autonomously build a map and localize against it in two important use cases: localization within the same colonoscopy or within different colonoscopies of the same patient. Code: https://github.com/jmorlana/ColonMapper.

Figures (6)

• Figure 1: ColonMapper builds a map from a sequence, localizing another sequence against it. Our Bayesian filter driven by deep global descriptors enables intra and cross-sequence localization.
• Figure 2: Topological maps from the withdrawal phase of Seq_027 in Endomapper (left) and from seq1 in C3VD (right).
• Figure 3: Precision-Recall curves for C3VD. Categories: Single-Image (SI), Bayesian (Bayes), Reject spurious (R).
• Figure 4: Precision-Recall curves for Endomapper. Categories: Single-Image (SI), Bayesian (Bayes), Reject spurious (R).
• Figure 5: Likelihood (top) and $p_{sum}$ (bottom) for R50-NV-H - Bayes + R in withdrawal_035 vs withdrawal_map_027. Labels: cecum, ascending, transverse, descending, sigmoid, rectum, retroflexure, none.