BronchoTrack: Airway Lumen Tracking for Branch-Level Bronchoscopic Localization

TL;DR

BronchoTrack tackles the essential problem of real-time bronchoscope localization by proposing a fast, generalizable branch-level localization pipeline that detects lumens, tracks multiple lumen objects, and maps observations to patient-specific airway anatomy through a semantic airway graph. The method combines a lightweight lumen detector (YOLOv7), multi-object tracking with Kalman-filter predictions and Re-ID-based appearance cues, and a training-free detection-airway association enabled by a semantic graph, augmented by an adapted loop-closure module for robust recovery. It is validated on nine patient datasets and in-vivo porcine experiments, showing strong localization accuracy up to the 4th-6th generations in offline data and up to the 8th generation in vivo, while maintaining real-time performance. The work offers practical significance by enabling branch-level localization without retraining for each patient, potentially improving intervention quality and enabling autonomous bronchoscopy.

Abstract

Localizing the bronchoscope in real time is essential for ensuring intervention quality. However, most existing methods struggle to balance between speed and generalization. To address these challenges, we present BronchoTrack, an innovative real-time framework for accurate branch-level localization, encompassing lumen detection, tracking, and airway association.To achieve real-time performance, we employ a benchmark lightweight detector for efficient lumen detection. We are the first to introduce multi-object tracking to bronchoscopic localization, mitigating temporal confusion in lumen identification caused by rapid bronchoscope movement and complex airway structures. To ensure generalization across patient cases, we propose a training-free detection-airway association method based on a semantic airway graph that encodes the hierarchy of bronchial tree structures.Experiments on nine patient datasets demonstrate BronchoTrack's localization accuracy of 85.64 \%, while accessing up to the 4th generation of airways.Furthermore, we tested BronchoTrack in an in-vivo animal study using a porcine model, where it successfully localized the bronchoscope into the 8th generation airway.Experimental evaluation underscores BronchoTrack's real-time performance in both satisfying accuracy and generalization, demonstrating its potential for clinical applications.

Figures (6)

• Figure 1: (a) Diagram of BronchoTrack. (b)(c)(d) are the modules of BronchoTrack. Based on motion models of tracked lumens, BronchoTrack predicts bounding box position. Combining with deep feature descriptor extracted by Re-ID branch, BronchoTrack matches new detections with previous tracklets to propagate airway labels. After building an airway subgraph with incomplete airway labels from tracklets, the labels are refined based on contextual information and anatomical constraints from pre-operative airway graph. Finally, the refined tracklets are used to generate the coarse bronchoscope localization precise to branch anatomical level. With BronchoTrack-LC extension, we search closed loops after initial tracklets association to detect and recover from tracking failure.
• Figure 2: Illustration of BronchoTrack's airway association. Here, we exemplify the labeling of observed child branches under $l_i$. 1. Leveraging inter-frame association, the label $l_i$ is initially assigned. 2. We determine observed branches by considering the likelihood of observing a child branch of $l_i$ based on a negative correlation between its angle against $l_i$. Subsequently, we generate a 2D graph of $l_i$'s child branches projected from the 3D airway graph. The 2D graph is further rotated based on the estimated bronchoscope roll angle. 3. Ultimately, the Hungarian Algorithm is employed to match the two graphs to associate tracklets with airway branches.
• Figure 3: Visualization results of BronchoTrack-LC compared with BoT-SORT. We demonstrate BronchoTrack-LC's ability to handle challenging cases, such as motion blur and occluded lumens, by selecting sequences from the detector's test set. The identical box color signifies the same identity.
• Figure 4: BronchoTrack-LC testing results in human data. (a) Localization accuracy at each visited branch. (b) Average precision of identifying each visible branch as its corresponding branch label. (c) Average localization error among each trajectory. Localization accuracy of each case is listed above the graph. * denotes training cases for detector.
• Figure 5: BronchoTrack testing results during animal experiment using porcine model. (a) the targets and driving path of each trial. (b) Localization accuracy at each visited branch. (c) Average precision of identifying each visible branch as its corresponding branch label. (d) Average localization error among each trajectory. Localization accuracy of each trajectory is listed above the graph.