Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Virtual Reality for Immersive Education in Orthopedic Surgery Digital Twins

Jonas Hein, Jan Grunder, Lilian Calvet, Frédéric Giraud, Nicola Alessandro Cavalcanti, Fabio Carrillo, Philipp Fürnstahl

TL;DR

SurgTwinVR is the first to propose a virtual dynamic 3D environment that is a clone of a real surgery, encompassing the entire surgical scene, including the surgeon, anatomical structures, surgical instruments, and the operating room.

Abstract

Virtual Reality technology, when integrated with Surgical Digital Twins (SDTs), offers significant potential in medical training and surgical planning. We present SurgTwinVR, a VR application that immerses users within an SDT and enables them to navigate a high-fidelity virtual replica of the surgical environment. SurgTwinVR is the first VR application to utilize a dynamic 3D environment that is a clone of a real surgery, encompassing the entire surgical scene, including the surgeon, anatomy, and instruments. Our system utilizes a SDT with important improvements for real-time rendering and features to showcase the potential benefits of such an application in surgical education.

Virtual Reality for Immersive Education in Orthopedic Surgery Digital Twins

TL;DR

SurgTwinVR is the first to propose a virtual dynamic 3D environment that is a clone of a real surgery, encompassing the entire surgical scene, including the surgeon, anatomical structures, surgical instruments, and the operating room.

Abstract

Virtual Reality technology, when integrated with Surgical Digital Twins (SDTs), offers significant potential in medical training and surgical planning. We present SurgTwinVR, a VR application that immerses users within an SDT and enables them to navigate a high-fidelity virtual replica of the surgical environment. SurgTwinVR is the first VR application to utilize a dynamic 3D environment that is a clone of a real surgery, encompassing the entire surgical scene, including the surgeon, anatomy, and instruments. Our system utilizes a SDT with important improvements for real-time rendering and features to showcase the potential benefits of such an application in surgical education.
Paper Structure (5 sections, 2 figures)

This paper contains 5 sections, 2 figures.

Table of Contents

  1. Introduction
  2. System Overview
  3. Creation of the SDT
  4. Real-time rendering
  5. Conclusion

Figures (2)

  • Figure 1: Schematic overview of the SDT's acquisition setup. Five ceiling-mounted Azure Kinect RGB-D cameras capture the motion of the surgeon. A FusionTrack 500 marker-based tracking system captures the trajectory of the surgical instrument.
  • Figure 2: The VR application allows the user to interact with a surgical drill, for example to learn the entry points and angles for pedicle drilling. The playback can be paused and the surgeon and instrument can be hidden temporarily. After placing the drill, the user can visually compare their drill pose to the senior surgeons drill pose.