SPAN: Spatial-Projection Alignment for Monocular 3D Object Detection

TL;DR

SPAN addresses the suboptimal monocular 3D detection arising from decoupled attribute regression by enforcing geometric coherence through Spatial Point Alignment and 3D-2D Projection Alignment. It projects 3D corners to obtain a differentiable, geometry-consistent loss regime, and stabilizes training with a Hierarchical Task Learning schedule. The approach yields consistent improvements on KITTI and Waymo benchmarks without adding inference cost, indicating strong potential for real-world monocular systems. Overall, SPAN demonstrates the value of explicit geometric regularization for improving spatial accuracy and projection consistency in monocular 3D perception.

Abstract

Existing monocular 3D detectors typically tame the pronounced nonlinear regression of 3D bounding box through decoupled prediction paradigm, which employs multiple branches to estimate geometric center, depth, dimensions, and rotation angle separately. Although this decoupling strategy simplifies the learning process, it inherently ignores the geometric collaborative constraints between different attributes, resulting in the lack of geometric consistency prior, thereby leading to suboptimal performance. To address this issue, we propose novel Spatial-Projection Alignment (SPAN) with two pivotal components: (i). Spatial Point Alignment enforces an explicit global spatial constraint between the predicted and ground-truth 3D bounding boxes, thereby rectifying spatial drift caused by decoupled attribute regression. (ii). 3D-2D Projection Alignment ensures that the projected 3D box is aligned tightly within its corresponding 2D detection bounding box on the image plane, mitigating projection misalignment overlooked in previous works. To ensure training stability, we further introduce a Hierarchical Task Learning strategy that progressively incorporates spatial-projection alignment as 3D attribute predictions refine, preventing early stage error propagation across attributes. Extensive experiments demonstrate that the proposed method can be easily integrated into any established monocular 3D detector and delivers significant performance improvements.

Figures (7)

• Figure 1: Previous methods typically neglect the geometric collaborative constraints between different attributes, resulting in spatial errors and projection misalignment. Our method aligns 3D corners and matches 3D projections with 2D boxes to improve detection accuracy and consistency.
• Figure 2: Overview of the proposed Spatial-Projection Alignment (SPAN). After a monocular detector predicts 2D and 3D attributes, SPAN adds two constrains: (a) Spatial Point Alignment that aligns predicted box corners with ground-truth in 3D space, and (b) 3D-2D Projection Alignment that constrains the projected 3D box to fit tightly inside the 2D detection. Hierarchical Task Learning controls the relative weights of these losses during training.
• Figure 3: Correspondence between the projected 3D box and the 2D bounding box. The 2D bounding box is shown as a green solid rectangle, while the projected region of the 3D box is outlined in orange. The minimal enclosing rectangle of the projection is indicated by a red dashed line. As the figure demonstrates, when the eight corner points of the 3D box are projected onto the 2D image plane, at least four of these points are guaranteed to lie on the four boundaries of the 2D bounding box.
• Figure 4: Illustration of the task hierarchy. The overall training process is divided into four sequential stages. Under the dynamic adjustment of Hierarchical Task Learning, a subsequent stage can only receive a significant loss weight once its pre-tasks have been trained to a stable state.
• Figure 5: Qualitative results on KITTI val set. (a) MonoDETR monodetr, (b) MonoDGP monodgp, and (c) MonoDGP (+SPAN). For each image set, the top row presents the camera-view visualization, while the bottom row offers the corresponding bird’s-eye view. Ground-truth bounding boxes are rendered in green, and predictions are shown in red. We also circle some objects to highlight the difference between the baseline model and our method.