Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

G-SHARP: Gaussian Surgical Hardware Accelerated Real-time Pipeline

Vishwesh Nath, Javier G. Tejero, Ruilong Li, Filippo Filicori, Mahdi Azizian, Sean D. Huver

TL;DR

G-SHARP introduces a commercially compatible, real-time surgical 3D reconstruction pipeline built on the GSplat differentiable Gaussian rasterizer. It targets deformable tissue with a HexPlane-based temporal deformation model and a two-stage training regime, achieving state-of-the-art accuracy while meeting real-time constraints and commercial licensing needs. The approach is deployed via a Holoscan SDK application that runs on edge NVIDIA hardware, enabling practical intraoperative visualization. Overall, the work advances deployable, high-fidelity AR-guided surgery by combining deformable Gaussian splatting with market-ready tooling and hardware integration.

Abstract

We propose G-SHARP, a commercially compatible, real-time surgical scene reconstruction framework designed for minimally invasive procedures that require fast and accurate 3D modeling of deformable tissue. While recent Gaussian splatting approaches have advanced real-time endoscopic reconstruction, existing implementations often depend on non-commercial derivatives, limiting deployability. G-SHARP overcomes these constraints by being the first surgical pipeline built natively on the GSplat (Apache-2.0) differentiable Gaussian rasterizer, enabling principled deformation modeling, robust occlusion handling, and high-fidelity reconstructions on the EndoNeRF pulling benchmark. Our results demonstrate state-of-the-art reconstruction quality with strong speed-accuracy trade-offs suitable for intra-operative use. Finally, we provide a Holoscan SDK application that deploys G-SHARP on NVIDIA IGX Orin and Thor edge hardware, enabling real-time surgical visualization in practical operating-room settings.

G-SHARP: Gaussian Surgical Hardware Accelerated Real-time Pipeline

TL;DR

G-SHARP introduces a commercially compatible, real-time surgical 3D reconstruction pipeline built on the GSplat differentiable Gaussian rasterizer. It targets deformable tissue with a HexPlane-based temporal deformation model and a two-stage training regime, achieving state-of-the-art accuracy while meeting real-time constraints and commercial licensing needs. The approach is deployed via a Holoscan SDK application that runs on edge NVIDIA hardware, enabling practical intraoperative visualization. Overall, the work advances deployable, high-fidelity AR-guided surgery by combining deformable Gaussian splatting with market-ready tooling and hardware integration.

Abstract

We propose G-SHARP, a commercially compatible, real-time surgical scene reconstruction framework designed for minimally invasive procedures that require fast and accurate 3D modeling of deformable tissue. While recent Gaussian splatting approaches have advanced real-time endoscopic reconstruction, existing implementations often depend on non-commercial derivatives, limiting deployability. G-SHARP overcomes these constraints by being the first surgical pipeline built natively on the GSplat (Apache-2.0) differentiable Gaussian rasterizer, enabling principled deformation modeling, robust occlusion handling, and high-fidelity reconstructions on the EndoNeRF pulling benchmark. Our results demonstrate state-of-the-art reconstruction quality with strong speed-accuracy trade-offs suitable for intra-operative use. Finally, we provide a Holoscan SDK application that deploys G-SHARP on NVIDIA IGX Orin and Thor edge hardware, enabling real-time surgical visualization in practical operating-room settings.

Paper Structure

This paper contains 13 sections, 1 equation, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. EndoNeRF and EndoNeRF–Pulling.
  4. Surgical-specific 3DGS design.
  5. GSplat in the context of differentiable splatting.
  6. Method & Datasets
  7. Dataset
  8. Method Overview
  9. Network Architecture
  10. Training Configuration
  11. Holoscan Application
  12. Results
  13. Future Direction

Figures (2)

  • Figure 1: Figure: Training (top) and rendering (bottom) pipelines for temporal Gaussian Splatting of surgical scenes. The training phase uses RGB, depth, and segmentation masks from all frames to learn 3D Gaussians and a temporal deformation network. The real-time rendering pipeline implements a Holoscan streaming architecture with custom operators: EndoNeRFLoaderOp streams camera poses and timestamps frame-by-frame, GsplatLoaderOp loads the trained checkpoint once at startup, GsplatRenderOp applies temporal deformation and performs differentiable rasterization, HolovizOp provides GPU-accelerated visualization, and ImageSaverOp logs outputs to disk.
  • Figure 2: Figure: Ground truth frames with tools are shown on the left and the reconstruction from G-SHARP are shown on the right for different chose frames