Revisiting Monocular 3D Object Detection with Depth Thickness Field

TL;DR

Addressing monocular 3D detection, the paper argues that limitations lie in depth representations rather than only depth accuracy. It introduces Depth Thickness Field (DTF), a scene-to-instance depth-adapted representation learned by Scene-Level Depth Retargeting (SDR) and refined by Instance-Level Spatial Refinement (ISR). SDR converts traditional depth cues into densely structured depth thickness fields, while ISR enhances voxel-space occupancy with instance guidance. Experiments on KITTI and Waymo show state-of-the-art results and strong generalization across depth estimators, highlighting the practical value of depth representation design for monocular 3D perception.

Abstract

Monocular 3D object detection is challenging due to the lack of accurate depth. However, existing depth-assisted solutions still exhibit inferior performance, whose reason is universally acknowledged as the unsatisfactory accuracy of monocular depth estimation models. In this paper, we revisit monocular 3D object detection from the depth perspective and formulate an additional issue as the limited 3D structure-aware capability of existing depth representations (e.g., depth one-hot encoding or depth distribution). To address this issue, we introduce a novel Depth Thickness Field approach to embed clear 3D structures of the scenes. Specifically, we present MonoDTF, a scene-to-instance depth-adapted network for monocular 3D object detection. The framework mainly comprises a Scene-Level Depth Retargeting (SDR) module and an Instance-Level Spatial Refinement (ISR) module. The former retargets traditional depth representations to the proposed depth thickness field, incorporating the scene-level perception of 3D structures. The latter refines the voxel space with the guidance of instances, enhancing the 3D instance-aware capability of the depth thickness field and thus improving detection accuracy. Extensive experiments on the KITTI and Waymo datasets demonstrate our superiority to existing state-of-the-art (SoTA) methods and the universality when equipped with different depth estimation models. The code will be available.

Figures (6)

• Figure 1: (a), (b), and (c) show VOXEL features from three different depth representations. (c) provides a more comprehensive and clearer scene structure feature. The last image surrounded by a dashed box represents instance-aware BEV features derived from (c), focusing on instance structure.
• Figure 2: The overall framework of our proposed MonoDTF. The input image is first sent to the backbone to extract the features and the depth map is transformed to depth one-hot encoding in parallel. The Scene-Level Depth Retargeting (SDR) module implicitly learns 3D scene structure features, retargeting depth one-hot encoding to depth thickness field (Sec. \ref{['subsec:SDR']}). The Instance-Level Spatial Refinement (ISR) module refines the voxel features with the guidance of instances, eliminating the ambiguity of 3D occupation (Sec. \ref{['subsec:ISR']}). Finally, the detector is applied to predict the 3D bounding boxes.
• Figure 3: Comparisons between different depth representations. The two curves indicate the confidence of the pixel on each depth bin, obtained from three different depth representations respectively. The purple curve represents a thick object, and the claret one represents a thin object. The depth thickness field has the unique ability to represent the object thickness along the camera ray.
• Figure 4: Thickness indicated by Depth Thickness Field along the camera ray.
• Figure 5: An slice example of occupancy map at a certain height. (a) OccupancyM3D takes voxels containing point clouds as positive samples (dark purple) and those passed through by camera ray and without point clouds as negative samples (gray). (b) Our method enhances (a) by incorporating voxels contained within the 3D boxes as positive samples (light purple).