SplatSSC: Decoupled Depth-Guided Gaussian Splatting for Semantic Scene Completion

TL;DR

SplatSSC introduces depth-guided initialization for sparse 3D Gaussian primitives and a Decoupled Gaussian Aggregator to robustly render semantic voxels from monocular images. By coupling a GMF-enabled depth branch with a group-wise cross-attention fusion and a two-stage training regime, the method achieves state-of-the-art SSC results on Occ-ScanNet while reducing latency and memory usage. The core innovations—depth priors for initialization, GMF for efficient fusion, and DGA with geometry/semantics decoupling plus Probability Scale Loss—address the long-standing issues of random initialization and outlier artifacts in object-centric 3D representations. Extensive ablations demonstrate the necessity and effectiveness of each component, highlighting improved geometric accuracy and semantic fidelity. These advances hold promise for real-time monocular scene understanding in indoor and potentially large-scale environments, with implications for embodied AI and robotics.

Abstract

Monocular 3D Semantic Scene Completion (SSC) is a challenging yet promising task that aims to infer dense geometric and semantic descriptions of a scene from a single image. While recent object-centric paradigms significantly improve efficiency by leveraging flexible 3D Gaussian primitives, they still rely heavily on a large number of randomly initialized primitives, which inevitably leads to 1) inefficient primitive initialization and 2) outlier primitives that introduce erroneous artifacts. In this paper, we propose SplatSSC, a novel framework that resolves these limitations with a depth-guided initialization strategy and a principled Gaussian aggregator. Instead of random initialization, SplatSSC utilizes a dedicated depth branch composed of a Group-wise Multi-scale Fusion (GMF) module, which integrates multi-scale image and depth features to generate a sparse yet representative set of initial Gaussian primitives. To mitigate noise from outlier primitives, we develop the Decoupled Gaussian Aggregator (DGA), which enhances robustness by decomposing geometric and semantic predictions during the Gaussian-to-voxel splatting process. Complemented with a specialized Probability Scale Loss, our method achieves state-of-the-art performance on the Occ-ScanNet dataset, outperforming prior approaches by over 6.3% in IoU and 4.1% in mIoU, while reducing both latency and memory cost by more than 9.3%.

Figures (10)

• Figure 1: Comparison with prior framework. (a) Recent transformer-based SSC frameworks start with a large set of randomly initialized Gaussian primitives, introducing redundancy. (b) Our framework starts with a compact yet targeted set of Gaussian primitives, guided by geometric priors.
• Figure 2: An overview of our proposed SplatSSC architecture. Given a single input image, our model employs two parallel branches: a trainable image encoder to extract multi-scale image features, and a frozen, pretrained Depth-Anything model to extract depth features. After a sampling step, both features are fed into the proposed Group-wise Multi-scale Fusion (GMF) block and a two-convolution layer depth head, yielding a refined feature map and a low-resolution depth map. These outputs are then lifted to initialize a set of 3D Gaussian primitives. Subsequently, the primitives are processed by a multi-stage encoder and finally passed to our Decoupled Gaussian Aggregator (DGA) to render the final semantic voxels.
• Figure 3: Illustration of the proposed GCA layer. The weight matrix $W_a$ is shared across different groups and scales, thus reducing memory consumption and computational cost to obtain the attention score.
• Figure 4: Illustration of the proposed DGA. While GF.agghuang2024gaussianformer and GF2.agghuang2025gaussianformer2 wrongly produces the "floaters" from outliers, our DGA remains robust, as the low occupancy probability directly suppresses its erroneous semantic contribution.
• Figure 5: Qualitative results on the Occ-ScanNet-mini dataset. Our method achieves superior performance in scene completion and target object recall compared to others.