ReliOcc: Towards Reliable Semantic Occupancy Prediction via Uncertainty Learning

TL;DR

ReliOcc is proposed, a method designed to enhance the reliability of camera-based occupancy networks by integrating hybrid uncertainty from individual voxels with sampling-based noise and relative voxels through mix-up learning, and an uncertainty-aware calibration strategy is devised to further enhance model reliability in offline mode.

Abstract

Vision-centric semantic occupancy prediction plays a crucial role in autonomous driving, which requires accurate and reliable predictions from low-cost sensors. Although having notably narrowed the accuracy gap with LiDAR, there is still few research effort to explore the reliability in predicting semantic occupancy from camera. In this paper, we conduct a comprehensive evaluation of existing semantic occupancy prediction models from a reliability perspective for the first time. Despite the gradual alignment of camera-based models with LiDAR in term of accuracy, a significant reliability gap persists. To addresses this concern, we propose ReliOcc, a method designed to enhance the reliability of camera-based occupancy networks. ReliOcc provides a plug-and-play scheme for existing models, which integrates hybrid uncertainty from individual voxels with sampling-based noise and relative voxels through mix-up learning. Besides, an uncertainty-aware calibration strategy is devised to further enhance model reliability in offline mode. Extensive experiments under various settings demonstrate that ReliOcc significantly enhances model reliability while maintaining the accuracy of both geometric and semantic predictions. Importantly, our proposed approach exhibits robustness to sensor failures and out of domain noises during inference.

Figures (7)

• Figure 1: (a) Overview of proposed ReliOcc. Besides the original objective of an occupancy network, we introduce an uncertainty estimation branch and supervise it with absolute and relative uncertainty learning losses. (b) Absolute uncertainty learning. Deterministic logits are replaced with ones sampled from predicted distributions. (c) Relative uncertainty learning. We leverage the relative relationships between uncertainty pairs to further enhance uncertainty learning.
• Figure 2: Visual results of the error map and uncertainty map from the prediction by SGN mei2023camera and ReliOcc. In the uncertainty map, a closer proximity to yellow indicates a higher level of uncertainty.
• Figure 3: The reliability diagrams of semantic calibration gaps from SGN mei2023camera and ReliOcc.
• Figure 4: The comparison in accuracy and reliability performance between SGN mei2023camera and ReliOcc under four out-of-domain conditions. Due to space constraints, we provide the semantic comparison here, while the geometric comparison is included in the supplemental material.
• Figure A1: The geometric comparison in accuracy and reliability performance between SGN mei2023camera and ReliOcc under four out-of-domain conditions.