Lightweight Spatial Embedding for Vision-based 3D Occupancy Prediction

TL;DR

LightOcc tackles the memory and latency bottlenecks of voxel-based 3D occupancy prediction by enriching BEV representations with height information via a Lightweight Spatial Embedding pipeline. It introduces Global Spatial Sampling to form Single-Channel Occupancy from multi-view depth and a Spatial-to-Channel mechanism to extract Tri-Perspective View embeddings, which are fused through a Lightweight TPV Interaction to produce a Spatial Embedding that augments BEV features. BEV-CutMix further enhances data diversity without corrupting occlusion information. On Occ3D-nuScenes, LightOcc achieves state-of-the-art occupancy accuracy while preserving BEV efficiency, demonstrating strong practical potential for real-time autonomous driving.

Abstract

Occupancy prediction has garnered increasing attention in recent years for its comprehensive fine-grained environmental representation and strong generalization to open-set objects. However, cumbersome voxel features and 3D convolution operations inevitably introduce large overheads in both memory and computation, obstructing the deployment of occupancy prediction approaches in real-time autonomous driving systems. Although some methods attempt to efficiently predict 3D occupancy from 2D Bird's-Eye-View (BEV) features through the Channel-to-Height mechanism, BEV features are insufficient to store all the height information of the scene, which limits performance. This paper proposes LightOcc, an innovative 3D occupancy prediction framework that leverages Lightweight Spatial Embedding to effectively supplement the height clues for the BEV-based representation while maintaining its deployability. Firstly, Global Spatial Sampling is used to obtain the Single-Channel Occupancy from multi-view depth distribution. Spatial-to-Channel mechanism then takes the arbitrary spatial dimension of Single-Channel Occupancy as the feature dimension and extracts Tri-Perspective Views (TPV) Embeddings by 2D convolution. Finally, TPV Embeddings will interact with each other by Lightweight TPV Interaction module to obtain the Spatial Embedding that is optimal supplementary to BEV features. Sufficient experimental results show that LightOcc significantly increases the prediction accuracy of the baseline and achieves state-of-the-art performance on the Occ3D-nuScenes benchmark.

Figures (7)

• Figure 1: Comparison of different occupancy prediction frameworks. BEV-based methods replace voxel features with BEV features for better deployability but lose a portion of height information. Our method utilizes a lightweight module to extract Spatial Embedding that effectively supplements height information to BEV features, enabling more accurate prediction.
• Figure 2: The overall architecture of LightOcc. Global Spatial Sampling gathers the predicted depth distribution into Single-Channel Occupancy, which is used to extract TPV Embeddings via Spatial-to-Channel mechanism. Lightweight TPV Interaction then fuses TPV Embeddings into Spatial Embedding and effectively supplements the lost height information to the BEV features.
• Figure 3: Spatial-to-Channel Mechanism
• Figure 4: Lightweight TPV Interaction
• Figure 5: Illustration of BEV-CutMix. Since the visibility mask is radially laid out from the center to the periphery. Cut the occupancy into 4 pieces from the center and randomly mix those pieces cross scenes do not cause wrong occlusion.