Deformable Gaussian Splatting for Efficient and High-Fidelity Reconstruction of Surgical Scenes

TL;DR

EH-SurGS is introduced, an efficient and high-fidelity reconstruction algorithm for deformable surgical scenes that incorporates the life cycle of 3D Gaussians, effectively capturing both regular and irreversible deformations, thus enhancing reconstruction quality and improving rendering speed.

Abstract

Efficient and high-fidelity reconstruction of deformable surgical scenes is a critical yet challenging task. Building on recent advancements in 3D Gaussian splatting, current methods have seen significant improvements in both reconstruction quality and rendering speed. However, two major limitations remain: (1) difficulty in handling irreversible dynamic changes, such as tissue shearing, which are common in surgical scenes; and (2) the lack of hierarchical modeling for surgical scene deformation, which reduces rendering speed. To address these challenges, we introduce EH-SurGS, an efficient and high-fidelity reconstruction algorithm for deformable surgical scenes. We propose a deformation modeling approach that incorporates the life cycle of 3D Gaussians, effectively capturing both regular and irreversible deformations, thus enhancing reconstruction quality. Additionally, we present an adaptive motion hierarchy strategy that distinguishes between static and deformable regions within the surgical scene. This strategy reduces the number of 3D Gaussians passing through the deformation field, thereby improving rendering speed. Extensive experiments demonstrate that our method surpasses existing state-of-the-art approaches in both reconstruction quality and rendering speed. Ablation studies further validate the effectiveness and necessity of our proposed components. We will open-source our code upon acceptance of the paper.

Figures (4)

• Figure 1: (a) Visualization of tissue shearing (yellow box) and static areas (red box) in a surgical scene. (b) Comparison of reconstruction quality (PSNR) and rendering speed (FPS) with state-of-the-art surgical scene reconstruction algorithms. Our method achieves state-of-the-art performance.
• Figure 2: Overview of EH-SurGS. It consists of two core modules: Deformation Modeling with Life Cycle (Sec. \ref{['sec:def']}) and Adaptive Motion Hierarchy Strategy (Sec. \ref{['sec:mask']}). EH-SurGS initializes the point cloud $P_0$ through back-projection based on the input RGB image, depth map, and surgical tool mask. $P_0$ is used to initialize 3D Gaussians to represent the canonical space. The Adaptive Motion Hierarchy Strategy is then applied to distinguish between deformable and static regions. For deformable regions, Deformation Modeling with Life Cycle is used to obtain the deformed Gaussians. Finally, RGB and depth are rendered through the differentiable tile rasterizer, and the loss is computed by comparing the rendered results with the inputs.
• Figure 3: Visualization of reconstruction results. The baselines show artifacts or blurriness in their reconstructions, while our approach achieves high-quality reconstruction performance.
• Figure 4: (a) The rendered RGB image using the w/o LC model; (b) The rendered RGB image using our proposed deformation model (Full); (c) The rendered depth map using the w/o LC-add model; (d) The rendered depth map using our proposed deformation model (Full). Regions of interest are highlighted using red and yellow boxes.