Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LGS: A Light-weight 4D Gaussian Splatting for Efficient Surgical Scene Reconstruction

Hengyu Liu, Yifan Liu, Chenxin Li, Wuyang Li, Yixuan Yuan

TL;DR

A Lightweight 4D Gaussian Splatting framework (LGS) is introduced that can liberate the efficiency bottlenecks of both rendering and storage for dynamic endoscopic reconstruction, and Deformation-Aware Pruning is proposed, to minimize the redundancy of Gaussian quantities.

Abstract

The advent of 3D Gaussian Splatting (3D-GS) techniques and their dynamic scene modeling variants, 4D-GS, offers promising prospects for real-time rendering of dynamic surgical scenarios. However, the prerequisite for modeling dynamic scenes by a large number of Gaussian units, the high-dimensional Gaussian attributes and the high-resolution deformation fields, all lead to serve storage issues that hinder real-time rendering in resource-limited surgical equipment. To surmount these limitations, we introduce a Lightweight 4D Gaussian Splatting framework (LGS) that can liberate the efficiency bottlenecks of both rendering and storage for dynamic endoscopic reconstruction. Specifically, to minimize the redundancy of Gaussian quantities, we propose Deformation-Aware Pruning by gauging the impact of each Gaussian on deformation. Concurrently, to reduce the redundancy of Gaussian attributes, we simplify the representation of textures and lighting in non-crucial areas by pruning the dimensions of Gaussian attributes. We further resolve the feature field redundancy caused by the high resolution of 4D neural spatiotemporal encoder for modeling dynamic scenes via a 4D feature field condensation. Experiments on public benchmarks demonstrate efficacy of LGS in terms of a compression rate exceeding 9 times while maintaining the pleasing visual quality and real-time rendering efficiency. LGS confirms a substantial step towards its application in robotic surgical services.

LGS: A Light-weight 4D Gaussian Splatting for Efficient Surgical Scene Reconstruction

TL;DR

A Lightweight 4D Gaussian Splatting framework (LGS) is introduced that can liberate the efficiency bottlenecks of both rendering and storage for dynamic endoscopic reconstruction, and Deformation-Aware Pruning is proposed, to minimize the redundancy of Gaussian quantities.

Abstract

The advent of 3D Gaussian Splatting (3D-GS) techniques and their dynamic scene modeling variants, 4D-GS, offers promising prospects for real-time rendering of dynamic surgical scenarios. However, the prerequisite for modeling dynamic scenes by a large number of Gaussian units, the high-dimensional Gaussian attributes and the high-resolution deformation fields, all lead to serve storage issues that hinder real-time rendering in resource-limited surgical equipment. To surmount these limitations, we introduce a Lightweight 4D Gaussian Splatting framework (LGS) that can liberate the efficiency bottlenecks of both rendering and storage for dynamic endoscopic reconstruction. Specifically, to minimize the redundancy of Gaussian quantities, we propose Deformation-Aware Pruning by gauging the impact of each Gaussian on deformation. Concurrently, to reduce the redundancy of Gaussian attributes, we simplify the representation of textures and lighting in non-crucial areas by pruning the dimensions of Gaussian attributes. We further resolve the feature field redundancy caused by the high resolution of 4D neural spatiotemporal encoder for modeling dynamic scenes via a 4D feature field condensation. Experiments on public benchmarks demonstrate efficacy of LGS in terms of a compression rate exceeding 9 times while maintaining the pleasing visual quality and real-time rendering efficiency. LGS confirms a substantial step towards its application in robotic surgical services.

Paper Structure

This paper contains 13 sections, 6 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Preliminaries
  4. Deformation-Aware Pruning
  5. Gaussian-Attribute Pruning
  6. Feature Field Condensation
  7. Optimization
  8. Experiments
  9. Experimental Settings
  10. Experimental Results
  11. Ablation Studies
  12. Conclusion
  13. Acknowledgement.

Figures (2)

  • Figure 1: LGS Overview consists of (a) Deformation-Aware Pruning, (b) Gaussian-Attribute Pruning, (c) Feature Field Condensation, and distillation for optimization.
  • Figure 2: Rendered images of previous methods and ours: PSNR reflects the quality of the shown image and Size reflects the used memory to store model.