HyGE-Occ: Hybrid View-Transformation with 3D Gaussian and Edge Priors for 3D Panoptic Occupancy Prediction

TL;DR

This paper addresses the challenge of dense 3D panoptic occupancy prediction from multi-view imagery by proposing HyGE-Occ, a BEV-based pipeline that fuses discretized depth unprojection with continuous Gaussian depth reasoning through a Hybrid View-Transformation Branch and enhances boundary delineation with an Edge Prediction Module. The method trains end-to-end with four losses, combining $L_{LSS}$, $L_G$, $L_{occ}$, and $L_{edge}$ in the total objective $L_{total}=\lambda_{LSS}L_{LSS}+\lambda_GL_G+\lambda_{edge}L_{edge}+L_{occ}$, delivering sharper instance boundaries and more coherent 3D geometry. Evaluated on Occ3D-nuScenes, HyGE-Occ achieves state-of-the-art performance, demonstrating that integrating continuous Gaussian priors with discretized depth and explicit edge guidance yields robust 3D panoptic occupancy suitable for autonomous perception pipelines.

Abstract

3D Panoptic Occupancy Prediction aims to reconstruct a dense volumetric scene map by predicting the semantic class and instance identity of every occupied region in 3D space. Achieving such fine-grained 3D understanding requires precise geometric reasoning and spatially consistent scene representation across complex environments. However, existing approaches often struggle to maintain precise geometry and capture the precise spatial range of 3D instances critical for robust panoptic separation. To overcome these limitations, we introduce HyGE-Occ, a novel framework that leverages a hybrid view-transformation branch with 3D Gaussian and edge priors to enhance both geometric consistency and boundary awareness in 3D panoptic occupancy prediction. HyGE-Occ employs a hybrid view-transformation branch that fuses a continuous Gaussian-based depth representation with a discretized depth-bin formulation, producing BEV features with improved geometric consistency and structural coherence. In parallel, we extract edge maps from BEV features and use them as auxiliary information to learn edge cues. In our extensive experiments on the Occ3D-nuScenes dataset, HyGE-Occ outperforms existing work, demonstrating superior 3D geometric reasoning.

Figures (6)

• Figure 1: HyGE-Occ fuses continuous and discrete depth representations into a hybrid BEV space and integrates an edge prior to enhance boundary cues, yielding robust 3DPOP.
• Figure 2: Overview of our proposed HyGE-Occ. Our model takes multi-view images as input and first extracts image features through a shared image backbone. The Hybrid View-Transformation Branch fuses a discretized and continuous Gaussian-based depth representation to form hybrid BEV features that combine spatial precision and geometric continuity. The resulting BEV features are further refined by a Edge Prediction Module, which predicts BEV-level edge maps optimised with pseudo edge labels computed from the semantic ground truth. These boundary-enhanced BEV representations are then decoded by the panoptic head, consisting of semantic and instance center branches, to produce the final 3D panoptic occupancy prediction.
• Figure 3: Hybrid View-Transformation Branch. The proposed module integrates both discrete and continuous depth representation through a blending module to form hybrid BEV features that combine geometric continuity with spatial precision.
• Figure 4: Qualitative comparison on the Occ3D-nuScenes validation set. Compared to the baseline Panoptic-FlashOcc, the proposed HyGE-Occ produces more accurate and coherent panoptic occupancy predictions. Our method better delineates semantic and instance boundaries, particularly in regions with dense object interactions and occlusions.
• Figure 5: Visualization of BEV features. Comparison between BEV features generated by the discretized LSS, continuous GaussianLSS, and our proposed hybrid representation.