SliceOcc: Indoor 3D Semantic Occupancy Prediction with Vertical Slice Representation

TL;DR

This work addresses indoor 3D semantic occupancy prediction under dense occlusions where traditional BEV/TPV representations struggle. It introduces SliceOcc, a transformer-based approach that uses a vertical slice representation with pairs of planes and attention-guided queries to project multi-view image features into local planes and fuse them into voxel-level features for occupancy prediction. Through planar cross-attention and sliced spatial cross-attention, SliceOcc achieves a strong RGB-camera baseline, attaining an $mIoU$ of $15.45 ext{%}$ across $81$ categories on EmbodiedScan, and showing competitive performance with depth-based methods. The method demonstrates substantial improvements in indoor 3D scene understanding and provides a practical, code-released solution for robust indoor occupancy reasoning.

Abstract

3D semantic occupancy prediction is a crucial task in visual perception, as it requires the simultaneous comprehension of both scene geometry and semantics. It plays a crucial role in understanding 3D scenes and has great potential for various applications, such as robotic vision perception and autonomous driving. Many existing works utilize planar-based representations such as Bird's Eye View (BEV) and Tri-Perspective View (TPV). These representations aim to simplify the complexity of 3D scenes while preserving essential object information, thereby facilitating efficient scene representation. However, in dense indoor environments with prevalent occlusions, directly applying these planar-based methods often leads to difficulties in capturing global semantic occupancy, ultimately degrading model performance. In this paper, we present a new vertical slice representation that divides the scene along the vertical axis and projects spatial point features onto the nearest pair of parallel planes. To utilize these slice features, we propose SliceOcc, an RGB camera-based model specifically tailored for indoor 3D semantic occupancy prediction. SliceOcc utilizes pairs of slice queries and cross-attention mechanisms to extract planar features from input images. These local planar features are then fused to form a global scene representation, which is employed for indoor occupancy prediction. Experimental results on the EmbodiedScan dataset demonstrate that SliceOcc achieves a mIoU of 15.45% across 81 indoor categories, setting a new state-of-the-art performance among RGB camera-based models for indoor 3D semantic occupancy prediction. Code is available at https://github.com/NorthSummer/SliceOcc.

Figures (7)

• Figure 1: Given a set of multi-view images from an indoor scene, SliceOcc vertically slices the scene to generate pairs of slice features, which are then interpolated to form voxel features for 3D semantic occupancy prediction.
• Figure 2: The framework of SliceOcc. SliceOcc consists of an image encoder and an occupancy decoder. We use an image backbone as the encoder to extract image features. In the occupancy decoder, we employ SSCA (sliced spatial cross attention) to enable interaction between the slice features and the image features, allowing for information exchange. We then design PCA (planar cross attention) to refine the slice features. Finally, we concatenate the slice features along the height dimension and make predictions for the entire scene's semantic occupancy.
• Figure 3: Qualitative performance on the EmbodiedScan multi-view occupancy prediction benchmark.
• Figure 4: Effect of increasing input views on SliceOcc performance.
• Figure 5: Additional statistics on the effect of varying slice number.