Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GeoBEV: Learning Geometric BEV Representation for Multi-view 3D Object Detection

Jinqing Zhang, Yanan Zhang, Yunlong Qi, Zehua Fu, Qingjie Liu, Yunhong Wang

TL;DR

GeoBEV addresses the geometric quality gap in BEV-based multi-view 3D object detection by introducing Radial-Cartesian BEV Sampling (RC-Sampling) to efficiently generate dense, high-resolution BEV features, alongside In-Box Label supervision and Centroid-Aware Inner Loss (CAI Loss) to inject authentic object geometry into BEV representations. RC-Sampling avoids heavy 3D voxel intermediates and cross-attention bottlenecks, enabling fast, dense BEV construction; In-Box Label with CAI Loss guides depth scores toward true object geometry and inner structure. On nuScenes, GeoBEV achieves state-of-the-art performance (e.g., about 66.3% NDS on the test set, with strong val results), demonstrating significant gains from the proposed components and demonstrating compatibility with existing BEV detectors. The work highlights practical benefits for real-world perception by improving geometric fidelity without extra parameters, and the methods generalize to other BEV-based detectors.

Abstract

Bird's-Eye-View (BEV) representation has emerged as a mainstream paradigm for multi-view 3D object detection, demonstrating impressive perceptual capabilities. However, existing methods overlook the geometric quality of BEV representation, leaving it in a low-resolution state and failing to restore the authentic geometric information of the scene. In this paper, we identify the drawbacks of previous approaches that limit the geometric quality of BEV representation and propose Radial-Cartesian BEV Sampling (RC-Sampling), which outperforms other feature transformation methods in efficiently generating high-resolution dense BEV representation to restore fine-grained geometric information. Additionally, we design a novel In-Box Label to substitute the traditional depth label generated from the LiDAR points. This label reflects the actual geometric structure of objects rather than just their surfaces, injecting real-world geometric information into the BEV representation. In conjunction with the In-Box Label, Centroid-Aware Inner Loss (CAI Loss) is developed to capture the inner geometric structure of objects. Finally, we integrate the aforementioned modules into a novel multi-view 3D object detector, dubbed GeoBEV, which achieves a state-of-the-art result of 66.2\% NDS on the nuScenes test set. The code is available at https://github.com/mengtan00/GeoBEV.git.

GeoBEV: Learning Geometric BEV Representation for Multi-view 3D Object Detection

TL;DR

GeoBEV addresses the geometric quality gap in BEV-based multi-view 3D object detection by introducing Radial-Cartesian BEV Sampling (RC-Sampling) to efficiently generate dense, high-resolution BEV features, alongside In-Box Label supervision and Centroid-Aware Inner Loss (CAI Loss) to inject authentic object geometry into BEV representations. RC-Sampling avoids heavy 3D voxel intermediates and cross-attention bottlenecks, enabling fast, dense BEV construction; In-Box Label with CAI Loss guides depth scores toward true object geometry and inner structure. On nuScenes, GeoBEV achieves state-of-the-art performance (e.g., about 66.3% NDS on the test set, with strong val results), demonstrating significant gains from the proposed components and demonstrating compatibility with existing BEV detectors. The work highlights practical benefits for real-world perception by improving geometric fidelity without extra parameters, and the methods generalize to other BEV-based detectors.

Abstract

Bird's-Eye-View (BEV) representation has emerged as a mainstream paradigm for multi-view 3D object detection, demonstrating impressive perceptual capabilities. However, existing methods overlook the geometric quality of BEV representation, leaving it in a low-resolution state and failing to restore the authentic geometric information of the scene. In this paper, we identify the drawbacks of previous approaches that limit the geometric quality of BEV representation and propose Radial-Cartesian BEV Sampling (RC-Sampling), which outperforms other feature transformation methods in efficiently generating high-resolution dense BEV representation to restore fine-grained geometric information. Additionally, we design a novel In-Box Label to substitute the traditional depth label generated from the LiDAR points. This label reflects the actual geometric structure of objects rather than just their surfaces, injecting real-world geometric information into the BEV representation. In conjunction with the In-Box Label, Centroid-Aware Inner Loss (CAI Loss) is developed to capture the inner geometric structure of objects. Finally, we integrate the aforementioned modules into a novel multi-view 3D object detector, dubbed GeoBEV, which achieves a state-of-the-art result of 66.2\% NDS on the nuScenes test set. The code is available at https://github.com/mengtan00/GeoBEV.git.
Paper Structure (24 sections, 7 equations, 5 figures, 7 tables)

This paper contains 24 sections, 7 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Depth Prediction Based BEV Representation
  4. Transformer Based BEV Representation
  5. Method
  6. Overall Architecture
  7. Radial-Cartesian BEV Sampling
  8. In-Box Label
  9. Centroid-Aware Inner Loss
  10. Experiments
  11. Dataset and Metrics
  12. Implementation Details
  13. Main Results
  14. Ablation Study
  15. Component Analysis
  16. ...and 9 more sections

Figures (5)

  • Figure 1: Comparison between BEV representations. BEVDepth is chosen as the baseline. Larger BEV size, RC-Sampling and In-Box Label are added in turn. The boxes represent the ground truth of the scene and brightness reveals the norm of the features. The background is filtered out to show the difference in the foreground.
  • Figure 2: Overall architecture of GeoBEV. The multi-view images are processed into the image features and depth scores. The depth scores are supervised by the In-Box Label that records authentic geometric structures of objects through the Centroid-Aware Inner Loss. Radial-Cartesian BEV Sampling then efficiently generates dense BEV representation with high resolution.
  • Figure 3: The illustration of Radial-Cartesian BEV Sampling. After high-dimensional matrix multiplication between the transposed image features and depth scores, the $H$ dimension is squeezed to obtain Radial BEV features.
  • Figure 4: Illustration of the associated design of In-Box Label. $H, W, D$ represent the height, width and depth dimensions. The boxes are the GT boxes. The red squares and black dots denote the positive and negative points of the In-Box Label. The blue crosses are the points that are not supervised. The deeper color in (d) means higher loss weight.
  • Figure 5: Visualization results on BEV representation of GeoBEV and its baseline. The background is filtered out to show the difference in the foreground. The red boxes and green boxes represent the ground truth and the predicted boxes, respectively. The dashed blue rectangles illustrate that the geometry guided BEV representations result in higher detection accuracy.