Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Omnidirectional Depth-Aided Occupancy Prediction based on Cylindrical Voxel for Autonomous Driving

Chaofan Wu, Jiaheng Li, Jinghao Cao, Ming Li, Yongkang Feng, Jiayu Wu Shuwen Xu, Zihang Gao, Sidan Du, Yang Li

TL;DR

This paper tackles the problem of accurate 3D occupancy prediction for autonomous driving under limited geometric priors by leveraging omnidirectional depth as a geometric prior and introducing a cylindrical (polar) voxel representation to match surround-view geometry. The authors propose OmniDepth-Occ, a Sketch-Coloring framework that uses depth priors to constrain a transformer-based occupancy predictor and maps 2D features onto a cylindrical voxel grid via cross-attention, aided by a 2D semantic head and temporal fusion. A virtual six-fisheye dataset is created in CARLA to address data gaps, providing extensive fisheye RGB and semantic labels in cylindrical voxels, with results showing significant improvements over baselines, especially in near-field regions. These contributions enable denser, more accurate 3D scene understanding for navigation and obstacle avoidance, with potential extensions to 3D reconstruction and object detection.

Abstract

Accurate 3D perception is essential for autonomous driving. Traditional methods often struggle with geometric ambiguity due to a lack of geometric prior. To address these challenges, we use omnidirectional depth estimation to introduce geometric prior. Based on the depth information, we propose a Sketch-Coloring framework OmniDepth-Occ. Additionally, our approach introduces a cylindrical voxel representation based on polar coordinate to better align with the radial nature of panoramic camera views. To address the lack of fisheye camera dataset in autonomous driving tasks, we also build a virtual scene dataset with six fisheye cameras, and the data volume has reached twice that of SemanticKITTI. Experimental results demonstrate that our Sketch-Coloring network significantly enhances 3D perception performance.

Omnidirectional Depth-Aided Occupancy Prediction based on Cylindrical Voxel for Autonomous Driving

TL;DR

This paper tackles the problem of accurate 3D occupancy prediction for autonomous driving under limited geometric priors by leveraging omnidirectional depth as a geometric prior and introducing a cylindrical (polar) voxel representation to match surround-view geometry. The authors propose OmniDepth-Occ, a Sketch-Coloring framework that uses depth priors to constrain a transformer-based occupancy predictor and maps 2D features onto a cylindrical voxel grid via cross-attention, aided by a 2D semantic head and temporal fusion. A virtual six-fisheye dataset is created in CARLA to address data gaps, providing extensive fisheye RGB and semantic labels in cylindrical voxels, with results showing significant improvements over baselines, especially in near-field regions. These contributions enable denser, more accurate 3D scene understanding for navigation and obstacle avoidance, with potential extensions to 3D reconstruction and object detection.

Abstract

Accurate 3D perception is essential for autonomous driving. Traditional methods often struggle with geometric ambiguity due to a lack of geometric prior. To address these challenges, we use omnidirectional depth estimation to introduce geometric prior. Based on the depth information, we propose a Sketch-Coloring framework OmniDepth-Occ. Additionally, our approach introduces a cylindrical voxel representation based on polar coordinate to better align with the radial nature of panoramic camera views. To address the lack of fisheye camera dataset in autonomous driving tasks, we also build a virtual scene dataset with six fisheye cameras, and the data volume has reached twice that of SemanticKITTI. Experimental results demonstrate that our Sketch-Coloring network significantly enhances 3D perception performance.

Paper Structure

This paper contains 19 sections, 6 equations, 7 figures, 7 tables.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. Occupancy Datasets
  4. 3D Scene Representation
  5. Vision-Based Occupancy Prediction
  6. Virtual Scene Occ Dateset with Fisheye Camera Input
  7. Methodology
  8. Overall Architecture
  9. Cylinder Sketch Module
  10. Coloring Module
  11. 2D Semantic Head
  12. Temporal Fusion
  13. Refine Module
  14. Training Loss
  15. Experiment
  16. ...and 4 more sections

Figures (7)

  • Figure 1: The typical workflow of occupancy prediction & our Sketch-Coloring framework.
  • Figure 2: Two instances to illustrate the characteristic of Cartesian and Cylinder coordinate system.
  • Figure 3: The distribution of cameras on our unmanned vehicle.
  • Figure 4: Overall architecture
  • Figure 5: The dilation process after the Cylinder Sketch Module
  • ...and 2 more figures