Improving Surgical Situational Awareness with Signed Distance Field: A Pilot Study in Virtual Reality

TL;DR

This work tackles the challenge of conveying actionable navigational guidance during image-guided surgery by introducing a modular, open-source pipeline that computes Signed Distance Fields ($SDF$) from segmented imaging data at initialization and uses these distances to drive multimodal feedback in VR. It implements visual, audio, and haptic modalities within the AMBF/FIVRS framework to indicate proximity to critical anatomy and to constrain instrument interactions. A pilot study with four clinicians performing mastoidectomy in VR shows that haptic feedback reduces unintended contacts and lowers workload, with audio feedback also improving safety, while visual feedback was less favorable. The approach demonstrates a generalizable, real-time guidance solution for CT/MRI-based procedures and sets the stage for larger, multi-procedure experiments and integration with robotic systems.

Abstract

The introduction of image-guided surgical navigation (IGSN) has greatly benefited technically demanding surgical procedures by providing real-time support and guidance to the surgeon during surgery. To develop effective IGSN, a careful selection of the surgical information and the medium to present this information to the surgeon is needed. However, this is not a trivial task due to the broad array of available options. To address this problem, we have developed an open-source library that facilitates the development of multimodal navigation systems in a wide range of surgical procedures relying on medical imaging data. To provide guidance, our system calculates the minimum distance between the surgical instrument and the anatomy and then presents this information to the user through different mechanisms. The real-time performance of our approach is achieved by calculating Signed Distance Fields at initialization from segmented anatomical volumes. Using this framework, we developed a multimodal surgical navigation system to help surgeons navigate anatomical variability in a skull base surgery simulation environment. Three different feedback modalities were explored: visual, auditory, and haptic. To evaluate the proposed system, a pilot user study was conducted in which four clinicians performed mastoidectomy procedures with and without guidance. Each condition was assessed using objective performance and subjective workload metrics. This pilot user study showed improvements in procedural safety without additional time or workload. These results demonstrate our pipeline's successful use case in the context of mastoidectomy.

Figures (7)

• Figure 1: Hardware setup for virtual drilling simulator. The hardware setup emulates a real mastoidectomy environment with the head-mounted display (HMD) in place of a stereo microscope, the haptic device in place of the surgical drill, and a foot pedal interface for actuating the drill. The setup is housed on a movable cart for portability.
• Figure 2: System Architecture. The proposed Surgical navigation plugin is developed on top of the FIVRS frameworkFIVRS2023. SDF calculation is done at the initialization phase using the same CT scan that is loaded into the simulation environment. At runtime, multimodal feedback is provided to the users via the haptic device, speakers, and head mounted display incorporated in the FIVRS system.
• Figure 3: Example visualization of an SDF volume's slice. (a) Segmented CT scan showing three anatomies: Temporomandibular (TMJ), Ear Canal (EAC), and Sinus, (b) SDF slice for EAC, (c) SDF slice for TMJ, and (d) SDF slice for Sinus. Voxels at each slice store the minimum distance between that voxel's location and a specific anatomy. The units for the color scale are $mm$. (e) Combined SDF image of TMJ, EAC, and Sinus. In the combined slice, different regions are color-coded by the closest anatomy (Green: EAC, Red: TMJ, Blue: Sinus).
• Figure 4: Sequence of snapshots (from 1 to 4) showing the Mastoidectomy procedure performed by the user study participants.
• Figure 5: Visual Feedback. Textual overlay provides the name of the closest anatomy and the distance to that specific anatomy via HMD in stereoscopic view.