Monocular Endoscopic Tissue 3D Reconstruction with Multi-Level Geometry Regularization

TL;DR

This work introduces surface-aware reconstruction, initially employing a Sign Distance Field-based method to construct a mesh, subsequently utilizing this mesh to constrain the Gaussian Splatting reconstruction process, and incorporates local rigidity and global non-rigidity restrictions to guide Gaussian deformation.

Abstract

Reconstructing deformable endoscopic tissues is crucial for achieving robot-assisted surgery. However, 3D Gaussian Splatting-based approaches encounter challenges in achieving consistent tissue surface reconstruction, while existing NeRF-based methods lack real-time rendering capabilities. In pursuit of both smooth deformable surfaces and real-time rendering, we introduce a novel approach based on 3D Gaussian Splatting. Specifically, we introduce surface-aware reconstruction, initially employing a Sign Distance Field-based method to construct a mesh, subsequently utilizing this mesh to constrain the Gaussian Splatting reconstruction process. Furthermore, to ensure the generation of physically plausible deformations, we incorporate local rigidity and global non-rigidity restrictions to guide Gaussian deformation, tailored for the highly deformable nature of soft endoscopic tissue. Based on 3D Gaussian Splatting, our proposed method delivers a fast rendering process and smooth surface appearances. Quantitative and qualitative analysis against alternative methodologies shows that our approach achieves solid reconstruction quality in both textures and geometries.

Figures (4)

• Figure 1: Our methodology begins with the key point detection, followed by neighborhood identification and mesh based Gaussian Splatting reconstruction. Subsequently, global and local restriction on the deformation is executed to obtain the final dynamic output.
• Figure 2: Comparison with other Gaussian Splatting-based methodologies. When we change the viewpoints, we clearly observe the 3D geometries of existing works are distorted, while our proposed framework reconstructs more consistent and much smoother endoscopic tissue surfaces. This demonstrates the usefulness of our proposed multi-level geometry regularization.
• Figure 3: Comparison with other methods on image quality and depth quality, where red boxes denote the incorrect reconstructed areas. Our method presents better reconstructed textures in the occluded regions.
• Figure 4: Detailed comparisons of the output RGB image are presented in this section. We emphasize regions rich in blood vessels and intricate texture. Our approach demonstrates a notable capability in preserving intricate details when contrasted with alternative methodologies. Furthermore, the inpainted regions in our method exhibit a higher degree of realism and coherence.