Reinforcement Learning for Follow-the-Leader Robotic Endoscopic Navigation via Synthetic Data

TL;DR

A follow-the-leader endoscopic robot based on a flexible continuum structure designed to minimize contact between the endoscope body and intestinal walls, thereby reducing patient discomfort and demonstrating the robustness and effectiveness of the proposed approach.

Abstract

Autonomous navigation is crucial for both medical and industrial endoscopic robots, enabling safe and efficient exploration of narrow tubular environments without continuous human intervention, where avoiding contact with the inner walls has been a longstanding challenge for prior approaches. We present a follow-the-leader endoscopic robot based on a flexible continuum structure designed to minimize contact between the endoscope body and intestinal walls, thereby reducing patient discomfort. To achieve this objective, we propose a vision-based deep reinforcement learning framework guided by monocular depth estimation. A realistic intestinal simulation environment was constructed in \textit{NVIDIA Omniverse} to train and evaluate autonomous navigation strategies. Furthermore, thousands of synthetic intraluminal images were generated using NVIDIA Replicator to fine-tune the Depth Anything model, enabling dense three-dimensional perception of the intestinal environment with a single monocular camera. Subsequently, we introduce a geometry-aware reward and penalty mechanism to enable accurate lumen tracking. Compared with the original Depth Anything model, our method improves $δ_{1}$ depth accuracy by 39.2% and reduces the navigation J-index by 0.67 relative to the second-best method, demonstrating the robustness and effectiveness of the proposed approach.

Figures (10)

• Figure 1: Reward formulation comparison. (a) and (b) share identical reward values; however, (b) is located closer to the colon wall. (c) illustrates our formulation, which explicitly accounts for previous motion errors.
• Figure 2: (a) 3D panorama of the FTL colonoscope robot, (b) FTL mechanism illustration.
• Figure 3: Overview of the proposed DepthColNet framework. (a) Fine-tuning pipeline based on Depth Anything, where adapter layers are embedded in Transformer blocks for endoscopic domain adaptation. (b) Details of the adapter and Colon Depth Enhancement Block (CDEB), while CDEB refines local geometry via depthwise separable convolution and channel attention.
• Figure 4: $C_0$, $C_1$ share the same texture, but $C_1$ shows a more complex path. $C_0$, $C_2$ have similar paths but differ in texture.
• Figure 5: The overview of our proposed vision-based deep reinforcement learning navigation framework.