NAVIUS: Navigated Augmented Reality Visualization for Ureteroscopic Surgery

TL;DR

Kidney stone disease affects a large population and ureteroscopy is challenged by a limited field of view. The authors introduce NAVIUS, an augmented reality navigation system that fuses preoperative CT-derived 3D kidney anatomy with real-time ureteroscope tracking and holographic overlays on the HoloLens 2, implemented through 3D Slicer, Unity, and OpenIGTLink. In phantom-based experiments with six surgeons, NAVIUS increased explored volume by 23.73% and reduced perceived workload by up to 27.27% (with SUS ~67), demonstrating feasible integration of AR navigation into ureteroscopy. These findings suggest NAVIUS can enhance thoroughness of exploration and surgeon experience, with potential to improve surgical outcomes and inform future multimodal AR-assisted surgical workflows.

Abstract

Ureteroscopy is the standard of care for diagnosing and treating kidney stones and tumors. However, current ureteroscopes have a limited field of view, requiring significant experience to adequately navigate the renal collecting system. This is evidenced by the fact that inexperienced surgeons have higher rates of missed stones. One-third of patients with residual stones require re-operation within 20 months. In order to aid surgeons to fully explore the kidney, this study presents the Navigated Augmented Reality Visualization for Ureteroscopic Surgery (NAVIUS) system. NAVIUS assists surgeons by providing 3D maps of the target anatomy, real-time scope positions, and preoperative imaging overlays. To enable real-time navigation and visualization, we integrate an electromagnetic tracker-based navigation pipeline with augmented reality visualizations. NAVIUS connects to 3D Slicer and Unity with OpenIGTLink, and uses HoloLens 2 as a holographic interface. We evaluate NAVIUS through a user study where surgeons conducted ureteroscopy on kidney phantoms with and without visual guidance. With our proposed system, we observed that surgeons explored more areas within the collecting system with NAVIUS (average 23.73% increase), and NASA-TLX metrics were improved (up to 27.27%). NAVIUS acts as a step towards better surgical outcomes and surgeons' experience. The codebase for the system will be available at: https://github.com/vu-maple-lab/NAVIUS.

Figures (5)

• Figure 1: Overall workflow of the system. We attach EM trackers to the ureteroscope and take CT scans of the phantoms. We create 3D models from CT scans and register the scope tip positions to CT coordinate plane using 3D Slicer. Scope positions and trail, CT slices, and 3D model are visualized in HoloLens 2. Letters refer to sections in the methods.
• Figure 2: Application features. (a) Complete anatomy visualizations that can be placed anywhere. (b) Alternatively, segmented collecting system can be used as visualization. Stones or other pathologies detected can be annotated with holographic objects of different colors. (c) Overlay of preoperative scans can be seen on holograms. (d) Scope tip position and tracked trajectory are visualized with a green marker and trail.
• Figure 3: (a) 3D printed collecting system. (b) Negative mold created with Ecoflex. (c) Wax model using mold. (d) Outer mold filling. (e) Boiling to remove wax model and create calyces. (f) Final phantom.
• Figure 4: Experimental setup (a) EM tracker attachment to scope. (b) Phantoms with registration fiducials. (c) Mock OR experimental setup. (d) HoloLens 2 field of view. (e) 3D Slicer view with registered scope positions and preoperative scans.
• Figure 5: Trajectories resulting from AR guidance (in orange) show in-depth exploration of the kidney phantom as opposed to without AR guidance (in blue). (A) and (B) shows trajectories created by four users exploring the kidney phantoms; 2 with AR guidance and 2 without.