Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Collaborative Extended Reality Prototype for 3D Surgical Planning and Visualization

Shi Qiu, Ruiyang Li, Qixuan Liu, Yuqi Tong, Yue Qiu, Yinqiao Wang, Yan Li, Chi-Wing Fu, Pheng-Ann Heng

TL;DR

The paper tackles the need for effective 3D, collaborative surgical planning beyond traditional 2D views by introducing a three-module XR prototype comprising MR-based immersive planning, cloud-based data management with role-based access, and a naked-eye light-field stereoscopic display for multi-device visualization. The system enables real-time planning, secure data sharing, and synchronized cross-device visualization to support clinical decision-making and patient communication. Evaluation with liver surgeons and engineering students demonstrates higher usability and perceived collaboration on the XR and coordinated platforms, indicating practical benefits for clinical workflows and education. The work lays a foundation for future integration of AI-assisted planning and broader device compatibility to enhance XR-enabled medical collaboration.

Abstract

We present a collaborative extended reality (XR) prototype for 3D surgical planning and visualization. Our system consists of three key modules: XR-based immersive surgical planning, cloud-based data management, and coordinated stereoscopic 3D displays for interactive visualization. We describe the overall workflow, core functionalities, implementations and setups. By conducting user studies on a liver resection surgical planning case, we demonstrate the effectiveness of our prototype and provide practical insights to inspire future advances in medical XR collaboration.

A Collaborative Extended Reality Prototype for 3D Surgical Planning and Visualization

TL;DR

The paper tackles the need for effective 3D, collaborative surgical planning beyond traditional 2D views by introducing a three-module XR prototype comprising MR-based immersive planning, cloud-based data management with role-based access, and a naked-eye light-field stereoscopic display for multi-device visualization. The system enables real-time planning, secure data sharing, and synchronized cross-device visualization to support clinical decision-making and patient communication. Evaluation with liver surgeons and engineering students demonstrates higher usability and perceived collaboration on the XR and coordinated platforms, indicating practical benefits for clinical workflows and education. The work lays a foundation for future integration of AI-assisted planning and broader device compatibility to enhance XR-enabled medical collaboration.

Abstract

We present a collaborative extended reality (XR) prototype for 3D surgical planning and visualization. Our system consists of three key modules: XR-based immersive surgical planning, cloud-based data management, and coordinated stereoscopic 3D displays for interactive visualization. We describe the overall workflow, core functionalities, implementations and setups. By conducting user studies on a liver resection surgical planning case, we demonstrate the effectiveness of our prototype and provide practical insights to inspire future advances in medical XR collaboration.
Paper Structure (4 sections, 2 figures, 2 tables)

This paper contains 4 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. System Details
  3. User Experience
  4. Conclusion

Figures (2)

  • Figure 1: XR-based surgical planning of liver resection. (a) Overview of the setup with an MR headset. (b) Mesh-based visualization of patient anatomy and the user interface panel. (c) Real-time computation of surgical safety metrics during plan determination. (d) Visualization of intersections between vessels and resection planes.
  • Figure 2: Naked-eye light field 3D display platform. (a) Coordinated visualization and interaction across multiple stereoscopic devices. (b) 3D rendering of liver anatomy with eight functional segments. (c) Delineation of the planned resection volume and liver remnant. (d) Simulation of surgical resection planes.