Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SAGS: Self-Adaptive Alias-Free Gaussian Splatting for Dynamic Surgical Endoscopic Reconstruction

Wenfeng Huang, Xiangyun Liao, Yinling Qian, Hao Liu, Yongming Yang, Wenjing Jia, Qiong Wang

TL;DR

SAGS tackles aliasing and non-rigid tissue motion in dynamic endoscopic 3D reconstruction by marrying 4D Gaussian Splatting with a self-adaptive, attention-based deformation decoder and explicit alias-free processing. The framework leverages HexPlane-encoded voxel representations, 3D smoothing, and 2D Mip filtering to suppress high-frequency artifacts while preserving fine tissue details. Empirical results on EndoNeRF and SCARED show consistent improvements in PSNR, SSIM, and LPIPS over prior methods, reflecting better geometric fidelity and perceptual quality. This approach holds promise for more reliable real-time visualization and planning in robot-assisted surgery.

Abstract

Surgical reconstruction of dynamic tissues from endoscopic videos is a crucial technology in robot-assisted surgery. The development of Neural Radiance Fields (NeRFs) has greatly advanced deformable tissue reconstruction, achieving high-quality results from video and image sequences. However, reconstructing deformable endoscopic scenes remains challenging due to aliasing and artifacts caused by tissue movement, which can significantly degrade visualization quality. The introduction of 3D Gaussian Splatting (3DGS) has improved reconstruction efficiency by enabling a faster rendering pipeline. Nevertheless, existing 3DGS methods often prioritize rendering speed while neglecting these critical issues. To address these challenges, we propose SAGS, a self-adaptive alias-free Gaussian splatting framework. We introduce an attention-driven, dynamically weighted 4D deformation decoder, leveraging 3D smoothing filters and 2D Mip filters to mitigate artifacts in deformable tissue reconstruction and better capture the fine details of tissue movement. Experimental results on two public benchmarks, EndoNeRF and SCARED, demonstrate that our method achieves superior performance in all metrics of PSNR, SSIM, and LPIPS compared to the state of the art while also delivering better visualization quality.

SAGS: Self-Adaptive Alias-Free Gaussian Splatting for Dynamic Surgical Endoscopic Reconstruction

TL;DR

SAGS tackles aliasing and non-rigid tissue motion in dynamic endoscopic 3D reconstruction by marrying 4D Gaussian Splatting with a self-adaptive, attention-based deformation decoder and explicit alias-free processing. The framework leverages HexPlane-encoded voxel representations, 3D smoothing, and 2D Mip filtering to suppress high-frequency artifacts while preserving fine tissue details. Empirical results on EndoNeRF and SCARED show consistent improvements in PSNR, SSIM, and LPIPS over prior methods, reflecting better geometric fidelity and perceptual quality. This approach holds promise for more reliable real-time visualization and planning in robot-assisted surgery.

Abstract

Surgical reconstruction of dynamic tissues from endoscopic videos is a crucial technology in robot-assisted surgery. The development of Neural Radiance Fields (NeRFs) has greatly advanced deformable tissue reconstruction, achieving high-quality results from video and image sequences. However, reconstructing deformable endoscopic scenes remains challenging due to aliasing and artifacts caused by tissue movement, which can significantly degrade visualization quality. The introduction of 3D Gaussian Splatting (3DGS) has improved reconstruction efficiency by enabling a faster rendering pipeline. Nevertheless, existing 3DGS methods often prioritize rendering speed while neglecting these critical issues. To address these challenges, we propose SAGS, a self-adaptive alias-free Gaussian splatting framework. We introduce an attention-driven, dynamically weighted 4D deformation decoder, leveraging 3D smoothing filters and 2D Mip filters to mitigate artifacts in deformable tissue reconstruction and better capture the fine details of tissue movement. Experimental results on two public benchmarks, EndoNeRF and SCARED, demonstrate that our method achieves superior performance in all metrics of PSNR, SSIM, and LPIPS compared to the state of the art while also delivering better visualization quality.

Paper Structure

This paper contains 19 sections, 13 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. 3D Reconstruction
  4. 3D Reconstruction for Medical Applications
  5. Methodology
  6. Preliminary of 3D Gaussian Splatting
  7. Point Cloud Information Acquisition
  8. 4D Representation for Deformable Tissue Reconstruction
  9. Self-adaptive Weighted Deformation Decoding
  10. Alias-Free Processing
  11. Experiments
  12. Datasets
  13. Evaluation Metrics
  14. Comparison with the State-of-the-Art Methods
  15. Ablation Study
  16. ...and 4 more sections

Figures (3)

  • Figure 1: The overall pipeline of the proposed SAGS framework. Depth maps from monocular and stereo estimation are re-projected to initialize 3D Gaussians, which are then refined using HexPlane encoding and a self-adaptive weighted deformation decoder MLP for deformation modeling. Alias-free filtering with Mip and smoothing filters is subsequently applied, and finally, rasterization generates high-fidelity rendered images and depth.
  • Figure 2: The qualitative result comparison between SOTAs and our proposed SAGS.
  • Figure 3: The qualitative result comparison between EndoGaussian and our proposed SAGS.