PANS: Probabilistic Airway Navigation System for Real-time Robust Bronchoscope Localization

TL;DR

PANS tackles the challenge of robust, real-time $6$-DOF bronchoscope localization in clinical settings by casting localization as a probabilistic tracking problem solved with a Monte Carlo particle filter. It integrates Depth-based Motion Inference (DMI) to propagate pose hypotheses using depth estimates trained on synthetic data, and Bronchial Semantic Analysis (BSA) to compute pose likelihoods via landmark-based bronchial branches and a centerline prior, including a Chamfer-like depth-to-structure metric. On 31 clinical cases, PANS achieves $ATE = 8.7 \pm 6.0$ mm and $SR10 = 70.0\%$, outperforming state-of-the-art depth-registration and monocular endoscopy methods, with ablations confirming the value of multiple visual representations and semantic landmarking for robustness in deep airway generations. The method runs in real time (≈15 Hz) and demonstrates generalization to unseen cases, suggesting significant potential to improve safety and effectiveness in pulmonary interventions.

Abstract

Accurate bronchoscope localization is essential for pulmonary interventions, by providing six degrees of freedom (DOF) in airway navigation. However, the robustness of current vision-based methods is often compromised in clinical practice, and they struggle to perform in real-time and to generalize across cases unseen during training. To overcome these challenges, we propose a novel Probabilistic Airway Navigation System (PANS), leveraging Monte-Carlo method with pose hypotheses and likelihoods to achieve robust and real-time bronchoscope localization. Specifically, our PANS incorporates diverse visual representations (\textit{e.g.}, odometry and landmarks) by leveraging two key modules, including the Depth-based Motion Inference (DMI) and the Bronchial Semantic Analysis (BSA). To generate the pose hypotheses of bronchoscope for PANS, we devise the DMI to accurately propagate the estimation of pose hypotheses over time. Moreover, to estimate the accurate pose likelihood, we devise the BSA module by effectively distinguishing between similar bronchial regions in endoscopic images, along with a novel metric to assess the congruence between estimated depth maps and the segmented airway structure. Under this probabilistic formulation, our PANS is capable of achieving the 6-DOF bronchoscope localization with superior accuracy and robustness. Extensive experiments on the collected pulmonary intervention dataset comprising 10 clinical cases confirm the advantage of our PANS over state-of-the-arts, in terms of both robustness and generalization in localizing deeper airway branches and the efficiency of real-time inference. The proposed PANS reveals its potential to be a reliable tool in the operating room, promising to enhance the quality and safety of pulmonary interventions.

Figures (4)

• Figure 1: Overview of the proposed PANS for real-time bronchoscope localization.
• Figure 2: Localization trajectory and error on an example patient case. Two inspection trajectories into the left and right lung are presented separately.
• Figure 3: (a) Localization ATE on different airway generations. (b) Localization ATE for the top two methods across various branches in an example case.
• Figure 4: Performance of PANS with different particle numbers.