Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UltimateDO: An Efficient Framework to Marry Occupancy Prediction with 3D Object Detection via Channel2height

Zichen Yu, Changyong Shu

TL;DR

This work proposes a method to achieve fast 3D object detection and occupancy prediction (UltimateDO), wherein the light occupancy prediction head in FlashOcc is married to 3D object detection network, with negligible additional timeconsuming of only 1.1ms while facilitating each other.

Abstract

Occupancy and 3D object detection are characterized as two standard tasks in modern autonomous driving system. In order to deploy them on a series of edge chips with better precision and time-consuming trade-off, contemporary approaches either deploy standalone models for individual tasks, or design a multi-task paradigm with separate heads. However, they might suffer from deployment difficulties (i.e., 3D convolution, transformer and so on) or deficiencies in task coordination. Instead, we argue that a favorable framework should be devised in pursuit of ease deployment on diverse chips and high precision with little time-consuming. Oriented at this, we revisit the paradigm for interaction between 3D object detection and occupancy prediction, reformulate the model with 2D convolution and prioritize the tasks such that each contributes to other. Thus, we propose a method to achieve fast 3D object detection and occupancy prediction (UltimateDO), wherein the light occupancy prediction head in FlashOcc is married to 3D object detection network, with negligible additional timeconsuming of only 1.1ms while facilitating each other. We instantiate UltimateDO on the challenging nuScenes-series benchmarks.

UltimateDO: An Efficient Framework to Marry Occupancy Prediction with 3D Object Detection via Channel2height

TL;DR

This work proposes a method to achieve fast 3D object detection and occupancy prediction (UltimateDO), wherein the light occupancy prediction head in FlashOcc is married to 3D object detection network, with negligible additional timeconsuming of only 1.1ms while facilitating each other.

Abstract

Occupancy and 3D object detection are characterized as two standard tasks in modern autonomous driving system. In order to deploy them on a series of edge chips with better precision and time-consuming trade-off, contemporary approaches either deploy standalone models for individual tasks, or design a multi-task paradigm with separate heads. However, they might suffer from deployment difficulties (i.e., 3D convolution, transformer and so on) or deficiencies in task coordination. Instead, we argue that a favorable framework should be devised in pursuit of ease deployment on diverse chips and high precision with little time-consuming. Oriented at this, we revisit the paradigm for interaction between 3D object detection and occupancy prediction, reformulate the model with 2D convolution and prioritize the tasks such that each contributes to other. Thus, we propose a method to achieve fast 3D object detection and occupancy prediction (UltimateDO), wherein the light occupancy prediction head in FlashOcc is married to 3D object detection network, with negligible additional timeconsuming of only 1.1ms while facilitating each other. We instantiate UltimateDO on the challenging nuScenes-series benchmarks.
Paper Structure (8 sections, 3 figures, 7 tables)

This paper contains 8 sections, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Framework
  4. Experiment
  5. Experimental Setup
  6. Comparison with State-of-the-art Methods
  7. Ablation Study
  8. Conclusion

Figures (3)

  • Figure 1: Different paradigm of interaction between 3D object detection and occupancy prediction. (a) performs occupancy prediction and 3D object detection separately. (b) shows the synchronous perception for detection and occupancy from shared voxel-level occupancy descriptor. (c) exhibits our UltimateDO where a light grafting occupancy module is married to 3D object detection. The abbreviation "Conv" represents convolution, "DA" denotes deformable convolution. Besides, the presence of "3D-DA", "3D-Conv" or "2D-Conv" in the icon indicates that the corresponding module is composed of these operators. Best viewed in color.
  • Figure 2: The diagram illustrates the overarching architecture of our proposed UltimateDO, which is best viewed in color and with zoom functionality. The region designated by the dashed box indicates the presence of replaceable modules. The light blue region corresponds to the optional temporal fusion module, and its utilization is contingent upon the activation of the red switch. MC is short for multi-convolution. Moreover, apart from the instructions provided for the three special icons located in the upper right corner, all remaining icons comply with the guidelines presented in Figure \ref{['fig:fig1']}.
  • Figure 3: Illustration of occupancy branch grafted on (a) BEV feature; (b) backbone of BEV encoder; (c) neck of BEV encoder. The abbreviation ”MH” represents MLP head, and ”CH” stands for CenterPoint head. Icon interpretation descriptions follows Figure. \ref{['fig:fig1']}. With the exception of the red line indicating the grafting location for occupancy branch, all other icons strictly adhere to the instructions depicted in Figure \ref{['fig:fig1']}.