Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Review of 3D Reconstruction Techniques for Deformable Tissues in Robotic Surgery

Mengya Xu, Ziqi Guo, An Wang, Long Bai, Hongliang Ren

TL;DR

This paper surveys and compares state-of-the-art 3D reconstruction methods for deformable tissues in robotic surgery, focusing on NeRF-based approaches (EndoNeRF, EndoSurf) and explicit Gaussian Splatting methods (LerPlane, 4D-GS). It provides a unified methodology overview, details implementation and training regimes, and evaluates real-time feasibility across three datasets (EndoNeRF, StereoMIS, C3VD), highlighting a domain gap that limits 4D-GS in surgical contexts. Key findings show NeRF-based methods achieve higher quality reconstructions at higher computational cost, LerPlane offers substantial speed advantages with competitive results on deformable scenes, and 4D-GS, despite real-time rendering potential, struggles due to surgical-domain differences. The study underscores the practical trade-offs between reconstruction fidelity, speed, and generalization, guiding future work toward domain-adaptive Gaussian representations and occlusion-aware sampling for real-time surgical scene reconstruction.

Abstract

As a crucial and intricate task in robotic minimally invasive surgery, reconstructing surgical scenes using stereo or monocular endoscopic video holds immense potential for clinical applications. NeRF-based techniques have recently garnered attention for the ability to reconstruct scenes implicitly. On the other hand, Gaussian splatting-based 3D-GS represents scenes explicitly using 3D Gaussians and projects them onto a 2D plane as a replacement for the complex volume rendering in NeRF. However, these methods face challenges regarding surgical scene reconstruction, such as slow inference, dynamic scenes, and surgical tool occlusion. This work explores and reviews state-of-the-art (SOTA) approaches, discussing their innovations and implementation principles. Furthermore, we replicate the models and conduct testing and evaluation on two datasets. The test results demonstrate that with advancements in these techniques, achieving real-time, high-quality reconstructions becomes feasible.

A Review of 3D Reconstruction Techniques for Deformable Tissues in Robotic Surgery

TL;DR

This paper surveys and compares state-of-the-art 3D reconstruction methods for deformable tissues in robotic surgery, focusing on NeRF-based approaches (EndoNeRF, EndoSurf) and explicit Gaussian Splatting methods (LerPlane, 4D-GS). It provides a unified methodology overview, details implementation and training regimes, and evaluates real-time feasibility across three datasets (EndoNeRF, StereoMIS, C3VD), highlighting a domain gap that limits 4D-GS in surgical contexts. Key findings show NeRF-based methods achieve higher quality reconstructions at higher computational cost, LerPlane offers substantial speed advantages with competitive results on deformable scenes, and 4D-GS, despite real-time rendering potential, struggles due to surgical-domain differences. The study underscores the practical trade-offs between reconstruction fidelity, speed, and generalization, guiding future work toward domain-adaptive Gaussian representations and occlusion-aware sampling for real-time surgical scene reconstruction.

Abstract

As a crucial and intricate task in robotic minimally invasive surgery, reconstructing surgical scenes using stereo or monocular endoscopic video holds immense potential for clinical applications. NeRF-based techniques have recently garnered attention for the ability to reconstruct scenes implicitly. On the other hand, Gaussian splatting-based 3D-GS represents scenes explicitly using 3D Gaussians and projects them onto a 2D plane as a replacement for the complex volume rendering in NeRF. However, these methods face challenges regarding surgical scene reconstruction, such as slow inference, dynamic scenes, and surgical tool occlusion. This work explores and reviews state-of-the-art (SOTA) approaches, discussing their innovations and implementation principles. Furthermore, we replicate the models and conduct testing and evaluation on two datasets. The test results demonstrate that with advancements in these techniques, achieving real-time, high-quality reconstructions becomes feasible.
Paper Structure (16 sections, 5 equations, 1 figure, 2 tables)

This paper contains 16 sections, 5 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Preliminaries
  4. NeRF: Neural Radiance Field
  5. 3D Gaussian Splatting
  6. EndoNeRF: Endoscopic NeRF Reconstruction
  7. EndoSurf: Endoscope-based Surface Reconstruction
  8. LerPlane: Linear Interpolation Plane
  9. 4D-GS: 4D Gaussian Splatting
  10. Experiments
  11. Dataset Description
  12. Implementation details
  13. Results and Evaluation
  14. Conclusion
  15. Acknowledgments.
  16. ...and 1 more sections

Figures (1)

  • Figure 1: Visualization Results of 4 models on $2$ datasets. EndoNeRF, EndoSurf, and LerPlane on the first dataset give good 3D results. Still, LerPlane on the StereoMIS dataset cannot restore the original color, and 4D-GS presents poor performance on both endoscopic datasets.