Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SurgicalGS: Dynamic 3D Gaussian Splatting for Accurate Robotic-Assisted Surgical Scene Reconstruction

Jialei Chen, Xin Zhang, Mobarakol Islam, Francisco Vasconcelos, Danail Stoyanov, Daniel S. Elson, Baoru Huang

TL;DR

This work presents SurgicalGS, a dynamic 3D Gaussian Splatting framework specifically designed for surgical scene reconstruction with improved geometric accuracy, using the Flexible Deformation Model to represent dynamic scene and introducing a normalised depth regularisation loss along with an unsupervised depth smoothness constraint to ensure more accurate geometric reconstruction.

Abstract

Accurate 3D reconstruction of dynamic surgical scenes from endoscopic video is essential for robotic-assisted surgery. While recent 3D Gaussian Splatting methods have shown promise in achieving high-quality reconstructions with fast rendering speeds, their use of inverse depth loss functions compresses depth variations. This can lead to a loss of fine geometric details, limiting their ability to capture precise 3D geometry and effectiveness in intraoperative application. To address these challenges, we present SurgicalGS, a dynamic 3D Gaussian Splatting framework specifically designed for surgical scene reconstruction with improved geometric accuracy. Our approach first initialises a Gaussian point cloud using depth priors, employing binary motion masks to identify pixels with significant depth variations and fusing point clouds from depth maps across frames for initialisation. We use the Flexible Deformation Model to represent dynamic scene and introduce a normalised depth regularisation loss along with an unsupervised depth smoothness constraint to ensure more accurate geometric reconstruction. Extensive experiments on two real surgical datasets demonstrate that SurgicalGS achieves state-of-the-art reconstruction quality, especially in terms of accurate geometry, advancing the usability of 3D Gaussian Splatting in robotic-assisted surgery.

SurgicalGS: Dynamic 3D Gaussian Splatting for Accurate Robotic-Assisted Surgical Scene Reconstruction

TL;DR

This work presents SurgicalGS, a dynamic 3D Gaussian Splatting framework specifically designed for surgical scene reconstruction with improved geometric accuracy, using the Flexible Deformation Model to represent dynamic scene and introducing a normalised depth regularisation loss along with an unsupervised depth smoothness constraint to ensure more accurate geometric reconstruction.

Abstract

Accurate 3D reconstruction of dynamic surgical scenes from endoscopic video is essential for robotic-assisted surgery. While recent 3D Gaussian Splatting methods have shown promise in achieving high-quality reconstructions with fast rendering speeds, their use of inverse depth loss functions compresses depth variations. This can lead to a loss of fine geometric details, limiting their ability to capture precise 3D geometry and effectiveness in intraoperative application. To address these challenges, we present SurgicalGS, a dynamic 3D Gaussian Splatting framework specifically designed for surgical scene reconstruction with improved geometric accuracy. Our approach first initialises a Gaussian point cloud using depth priors, employing binary motion masks to identify pixels with significant depth variations and fusing point clouds from depth maps across frames for initialisation. We use the Flexible Deformation Model to represent dynamic scene and introduce a normalised depth regularisation loss along with an unsupervised depth smoothness constraint to ensure more accurate geometric reconstruction. Extensive experiments on two real surgical datasets demonstrate that SurgicalGS achieves state-of-the-art reconstruction quality, especially in terms of accurate geometry, advancing the usability of 3D Gaussian Splatting in robotic-assisted surgery.

Paper Structure

This paper contains 19 sections, 14 equations, 4 figures, 3 tables.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. Stereo-Matching for MIS
  4. NeRF-based in Surgery
  5. 3DGS Applications in Surgery
  6. Methodology
  7. Preliminaries
  8. 3D Gaussian Splatting
  9. Dynamic Representation
  10. Proposed Method: SurgicalGS
  11. Dense Initialisation with Depth Priors
  12. Normalised Depth Regularisation
  13. Unsupervised Depth Smoothness
  14. Total Loss Function
  15. Experiments and Results
  16. ...and 4 more sections

Figures (4)

  • Figure 1: We propose a dynamic surgical scene reconstruction frame. Comparisons with EndoGaussian liu2024endogaussian and Deform3DGS yang2024deform3dgs demonstrate that SurgicalGS achieves superior performance in geometric reconstruction.
  • Figure 2: Overall of the proposed framework. Starting from an image and depth sequence, we associate 3D points between frames to initialise the Gaussian point cloud. Then, given a specific time $t$, the attributes of the Gaussians are modified by the deformation model and rendered as a 2D image and depth map. Finally, colour and depth loss are employed to refine results.
  • Figure 3: Visualisation of the 3D reconstruction results. Further details can be seen by zooming in.
  • Figure 4: Visualisation of ablation study on depth loss using the StereoMIS dataset. Further details can be seen by zooming in.