EndoGaussians: Single View Dynamic Gaussian Splatting for Deformable Endoscopic Tissues Reconstruction

TL;DR

EndoGaussians tackles the challenge of accurate 3D reconstruction of deformable endoscopic tissues from limited endoscopic views. It introduces a dynamic Gaussian Splatting framework paired with a two-stage pipeline: (i) Endoscopic Video Inpainting using a Flow-Guided Transformer to remove instruments, and (ii) single-view RGBD-based Gaussian Splatting with differentiable depth rasterization, depth regularization, and explicit hallucination control. The method includes a dense point-cloud initialization, a suite of depth and geometric losses, and a hallucination identification mechanism to curb spurious reconstructions. Across EndoNeRF and SCARED datasets, EndoGaussians achieves state-of-the-art PSNR, SSIM, and LPIPS with favorable reconstruction speed, offering a more trustworthy and practical tool for VR surgery and medical image analysis.

Abstract

The accurate 3D reconstruction of deformable soft body tissues from endoscopic videos is a pivotal challenge in medical applications such as VR surgery and medical image analysis. Existing methods often struggle with accuracy and the ambiguity of hallucinated tissue parts, limiting their practical utility. In this work, we introduce EndoGaussians, a novel approach that employs Gaussian Splatting for dynamic endoscopic 3D reconstruction. This method marks the first use of Gaussian Splatting in this context, overcoming the limitations of previous NeRF-based techniques. Our method sets new state-of-the-art standards, as demonstrated by quantitative assessments on various endoscope datasets. These advancements make our method a promising tool for medical professionals, offering more reliable and efficient 3D reconstructions for practical applications in the medical field.

Figures (1)

• Figure 1: Reconstructed image and depth result of our methods.