Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Panoptic-FlashOcc: An Efficient Baseline to Marry Semantic Occupancy with Panoptic via Instance Center

Zichen Yu, Changyong Shu, Qianpu Sun, Yifan Bian, Xiaobao Wei, Jiangyong Yu, Zongdai Liu, Dawei Yang, Hui Li, Yan Chen

TL;DR

Panoptic-FlashOcc is proposed, a straightforward yet robust 2D feature framework that enables realtime panoptic occupancy and simultaneously learns semantic occupancy and class-aware instance clustering in a single network.

Abstract

Panoptic occupancy poses a novel challenge by aiming to integrate instance occupancy and semantic occupancy within a unified framework. However, there is still a lack of efficient solutions for panoptic occupancy. In this paper, we propose Panoptic-FlashOcc, a straightforward yet robust 2D feature framework that enables realtime panoptic occupancy. Building upon the lightweight design of FlashOcc, our approach simultaneously learns semantic occupancy and class-aware instance clustering in a single network, these outputs are jointly incorporated through panoptic occupancy procession for panoptic occupancy. This approach effectively addresses the drawbacks of high memory and computation requirements associated with three-dimensional voxel-level representations. With its straightforward and efficient design that facilitates easy deployment, Panoptic-FlashOcc demonstrates remarkable achievements in panoptic occupancy prediction. On the Occ3D-nuScenes benchmark, it achieves exceptional performance, with 38.5 RayIoU and 29.1 mIoU for semantic occupancy, operating at a rapid speed of 43.9 FPS. Furthermore, it attains a notable score of 16.0 RayPQ for panoptic occupancy, accompanied by a fast inference speed of 30.2 FPS. These results surpass the performance of existing methodologies in terms of both speed and accuracy. The source code and trained models can be found at the following github repository: https://github.com/Yzichen/FlashOCC.

Panoptic-FlashOcc: An Efficient Baseline to Marry Semantic Occupancy with Panoptic via Instance Center

TL;DR

Panoptic-FlashOcc is proposed, a straightforward yet robust 2D feature framework that enables realtime panoptic occupancy and simultaneously learns semantic occupancy and class-aware instance clustering in a single network.

Abstract

Panoptic occupancy poses a novel challenge by aiming to integrate instance occupancy and semantic occupancy within a unified framework. However, there is still a lack of efficient solutions for panoptic occupancy. In this paper, we propose Panoptic-FlashOcc, a straightforward yet robust 2D feature framework that enables realtime panoptic occupancy. Building upon the lightweight design of FlashOcc, our approach simultaneously learns semantic occupancy and class-aware instance clustering in a single network, these outputs are jointly incorporated through panoptic occupancy procession for panoptic occupancy. This approach effectively addresses the drawbacks of high memory and computation requirements associated with three-dimensional voxel-level representations. With its straightforward and efficient design that facilitates easy deployment, Panoptic-FlashOcc demonstrates remarkable achievements in panoptic occupancy prediction. On the Occ3D-nuScenes benchmark, it achieves exceptional performance, with 38.5 RayIoU and 29.1 mIoU for semantic occupancy, operating at a rapid speed of 43.9 FPS. Furthermore, it attains a notable score of 16.0 RayPQ for panoptic occupancy, accompanied by a fast inference speed of 30.2 FPS. These results surpass the performance of existing methodologies in terms of both speed and accuracy. The source code and trained models can be found at the following github repository: https://github.com/Yzichen/FlashOCC.
Paper Structure (17 sections, 2 equations, 4 figures, 5 tables, 1 algorithm)

This paper contains 17 sections, 2 equations, 4 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Architecture
  4. Overview Architecture
  5. Semantic Occupancy Prediction
  6. Centerness Head
  7. Center Heatmap Head.
  8. Center Regression Head.
  9. Panoptic Occupancy Processing
  10. Instance Center Proposal.
  11. Nearest Assign.
  12. Experiment
  13. Experimental Setup
  14. Comparison with State-of-the-art Methods
  15. Ablation Study
  16. ...and 2 more sections

Figures (4)

  • Figure 1: Trade-off between Speed and Accuracy. The blue point plots are derived from reports in SparseOcc liu2023sparseocc, while the yellow ones are obtained through our own testing with same setting, i.e., the frames per second (FPS) were conducted using a A100 with PyTorch FP32 backend.
  • Figure 2: The overall architecture of our Panoptic-FlashOcc. The BEV generation and semantic occupancy prediction directly inherit the model structure from FlashOcc, while the newly added centerness head and panoptic occupancy processing enhances semantic occupancy prediction to panoramic occupancy prediction. Best viewed in color and zoom.
  • Figure 3: Qualitative results of panoptic occupancy on Occ3D-nuScenes.
  • Figure 4: Qualitative results of semantic occupancy on Occ3D-nuScenes.