Mixed Reality Guidance of a Surgical Scalpel Using Magic Leap: Evaluation on a 3D-Printed Liver Phantom

TL;DR

This paper addresses the need for precise intraoperative guidance in liver surgery by leveraging mixed reality with the Magic Leap headset to project a planned resection path onto a 3D-printed liver phantom. The authors develop a three-stage MR workflow—preoperative planning, registration via spatial mapping and fiducials, and real-time holographic guidance—and demonstrate that MR guidance reduces path deviation from $5.0$ mm to $2.0$ mm and shortens task time from $55$ s to $32$ s in a phantom study. Ten participants (residents and surgeons) showed improved accuracy and efficiency with MR guidance, along with positive subjective feedback on depth perception and confidence. The work establishes feasibility and potential clinical relevance while acknowledging limitations such as organ deformation, registration drift, and the need for robust intraoperative updates, outlining clear directions for future clinical translation and broader applicability.

Abstract

Augmented and mixed reality (MR) systems have the potential to improve surgical precision by overlaying digital guidance directly onto the operative field. This paper presents a novel MR guidance system using the Magic Leap head-mounted display to assist surgeons in executing precise scalpel movements during liver surgery. The system projects holographic cues onto a patient-specific 3D-printed liver phantom, guiding resection along a predetermined path. We describe the system design, including preoperative modeling, registration of virtual content to the phantom, and real-time visualization through the Magic Leap device. In a controlled phantom study, surgical trainees performed resection tasks with and without MR guidance. Quantitative results demonstrated that MR guidance improved cutting accuracy (mean deviation from planned path was reduced from 5.0 mm without AR to 2.0 mm with AR guidance) and efficiency (mean task time decreased from 55 s to 32 s). These improvements of approximately 60% in accuracy and 40% in speed underscore the potential benefit of MR in surgical navigation. Participants reported that the Magic Leap visualization enhanced depth perception and confidence in locating tumor boundaries. This work provides a comprehensive evaluation of an MR-assisted surgical guidance approach, highlighting its feasibility on a realistic organ phantom. We discuss the technical challenges (registration accuracy, line-of-sight, user ergonomics) and outline future steps toward clinical translation. The results suggest that Magic Leap-based MR guidance can significantly augment a surgeon's performance in delicate resection tasks, paving the way for safer and more precise liver surgery.

Figures (4)

• Figure 1: System architecture for Magic Leap-based mixed reality (MR) guidance. A patient-specific 3D liver model with tumor is used to plan the resection path preoperatively. The model and planned cut path are loaded into a Magic Leap application. During surgery, an optical tracking system (or Magic Leap’s spatial mapping) registers the physical 3D-printed liver phantom to the virtual model. The Magic Leap headset then projects the holographic model and cutting guide onto the surgeon’s view of the phantom in real time, allowing the surgeon to follow the planned path with the scalpel.
• Figure 2: Conceptual illustration of the MR guidance overlay on the liver phantom. The planned resection line (blue dashed curve) is projected onto the phantom’s surface, encircling the tumor (red area). The surgeon aligns the scalpel with this holographic line to follow the preplanned trajectory. In the actual MR view (through the Magic Leap), the liver surface and tumor are visualized in 3D with proper depth cues, aiding the surgeon in maintaining the correct path and ensuring the tumor is fully resected with margin.
• Figure 3: Comparison of cutting accuracy with and without MR guidance. The bar heights indicate the mean deviation (in mm) of the actual cut path from the planned resection path (lower is better). Error bars show $\pm 1$ standard deviation. MR guidance with Magic Leap significantly reduced the deviation, indicating surgeons cut much closer to the intended path when the holographic overlay was present.
• Figure 4: Task completion time for tumor resection with and without MR guidance. MR guidance reduced mean completion time from 55 s to 32 s. Error bars indicate $\pm 1$ SD across participants.