InnerGS: Internal Scenes Reconstruction and Segmentation via Factorized 3D Gaussian Splatting

TL;DR

This work introduces InnerGS, a novel extension of 3D Gaussian Splatting designed for internal volumetric reconstruction from sliced data without camera poses. It formulates an inner Gaussian density by evaluating per-slice contributions and implements a slice-based rendering pipeline with two sampling schemes—3D Ellipsoid Projection and Conditional Splatting—to efficiently reconstruct interior structures and synthesize arbitrary-depth views. The method is demonstrated on static and dynamic MRI data, achieving high PSNR/SSIM and enabling 4D reconstruction, as well as strong open-vocabulary segmentation when integrated with Vision-Language Models. By providing a plug-in CUDA implementation and modality-agnostic processing, InnerGS offers a practical, efficient framework for interior scene understanding with potential clinical impact in diagnostics and surgical planning.

Abstract

3D Gaussian Splatting (3DGS) has recently gained popularity for efficient scene rendering by representing scenes as explicit sets of anisotropic 3D Gaussians. However, most existing work focuses primarily on modeling external surfaces. In this work, we target the reconstruction of internal scenes, which is crucial for applications that require a deep understanding of an object's interior. By directly modeling a continuous volumetric density through the inner 3D Gaussian distribution, our model effectively reconstructs smooth and detailed internal structures from sparse sliced data. Beyond high-fidelity reconstruction, we further demonstrate the framework's potential for downstream tasks such as segmentation. By integrating language features, we extend our approach to enable text-guided segmentation of medical scenes via natural language queries. Our approach eliminates the need for camera poses, is plug-and-play, and is inherently compatible with any data modalities. We provide cuda implementation at: https://github.com/Shuxin-Liang/InnerGS.

Figures (10)

• Figure 1: Illustration of the two sampling method for rasterization: (Top) 3D Ellipsoid Projection computes a cube and project identical bounding boxes across slices, while (Bottom) Conditional Splatting adapts the bounding box per slice based on conditional Gaussians.
• Figure 2: Brain MR reconstructions in axial (top) and sagittal (bottom) views. The top row shows test set renderings with PSNR values overlaid. Red boxes highlight regions for detailed reconstruction. In each zoom-in region, the left patch is the ground truth, and the right is the prediction, with L1 errors indicating absolute differences within the region.
• Figure 3: Cardiac MR reconstructions (coronal view).
• Figure 4: Wrist MR reconstructions at Time = 16 across all axial slices with per-slice PSNR values.
• Figure 5: Wrist motion MR reconstructions across test timestamps for two axial slices: top — slice 1, bottom — slice 5. For each slice, predicted reconstructions (pred) are compared with ground-truth images (gt). Our method captures temporally coherent anatomical motion across both shallow and deep wrist structures.