Unleashing the Power of Chain-of-Prediction for Monocular 3D Object Detection

TL;DR

This work tackles monocular 3D object detection by addressing depth ambiguity through explicit modelling of inter-attribute correlations among depth, size, and orientation. It introduces Chain-of-Prediction (CoP) to learn feature-level cross-attribute dependencies in a progressive, residual-aggregated sequence, paired with an Uncertainty-Guided Selector (GS) that dynamically routes each object to either CoP or parallel prediction based on depth uncertainty. The combination yields state-of-the-art results on KITTI, nuScenes, and Waymo, with notable improvements in depth accuracy for distant objects and a favorable efficiency profile. The approach advances robust monocular 3D perception by adaptively balancing correlation exploitation and independence, guided by uncertainty estimates.

Abstract

Monocular 3D detection (Mono3D) aims to infer 3D bounding boxes from a single RGB image. Without auxiliary sensors such as LiDAR, this task is inherently ill-posed since the 3D-to-2D projection introduces depth ambiguity. Previous works often predict 3D attributes (e.g., depth, size, and orientation) in parallel, overlooking that these attributes are inherently correlated through the 3D-to-2D projection. However, simply enforcing such correlations through sequential prediction can propagate errors across attributes, especially when objects are occluded or truncated, where inaccurate size or orientation predictions can further amplify depth errors. Therefore, neither parallel nor sequential prediction is optimal. In this paper, we propose MonoCoP, an adaptive framework that learns when and how to leverage inter-attribute correlations with two complementary designs. A Chain-of-Prediction (CoP) explores inter-attribute correlations through feature-level learning, propagation, and aggregation, while an Uncertainty-Guided Selector (GS) dynamically switches between CoP and parallel paradigms for each object based on the predicted uncertainty. By combining their strengths, MonoCoP achieves state-of-the-art (SOTA) performance on KITTI, nuScenes, and Waymo, significantly improving depth accuracy, particularly for distant objects.

Figures (9)

• Figure 1: Overview of prediction paradigms in Mono3D. (a) Parallel Prediction: predicts multiple $3$D attributes (e.g., size, orientation, depth) independently, ignoring their inter-dependencies. (b) Sequential Prediction: predicts attributes step by step, conditioning each on the previously estimated ones, which easily causes error accumulation across attributes. (c) Chain-of-Prediction (Ours): captures feature-level inter-attribute correlations by progressively learning, propagating, and aggregating attribute-specific features, effectively mitigating error accumulation in b. (d) Dynamic Prediction (Ours): dynamically switches between CoP and parallel prediction for each object based on the predicted uncertainty, effectively combining strengths from both prediction paradigms.
• Figure 2: Illustration of inter-correlated $3$D attributes in Mono3D. Through the $3$D-to-$2$D projection, attributes such as depth, size, and orientation jointly determine an object’s appearance in the image, making them inherently coupled. As shown in Images 1–2, cars at different depths appear with similar $2$D sizes when their $3$D sizes differ, while in Images 2–3, the same car at a fixed depth exhibits apparent scale changes under different orientations. This projection-induced coupling leads to inherent ambiguity when inferring $3$D structure from a single $2$D image, highlighting the need to explicitly model their inter-correlations.
• Figure 3: MonoCoP Overview.$3$D attributes (e.g., depth, size, and orientation) are correlated through the $3$D-to-$2$D projection. MonoCoP learns when and how to exploit these correlations through two complementary modules. The Chain-of-Prediction (CoP) captures cross-attribute dependencies at the feature level, progressively propagating and aggregating attribute-specific cues to enhance geometric consistency and mitigate error accumulation. The Uncertainty-Guided Selector (GS) adaptively selects between CoP and parallel pathways for each object based on its depth uncertainty, combining their strengths to achieve more accurate and robust $3$D detection.
• Figure 4: Mean Absolute Error (MAE) on KITTI Val. We compute the MAE for predicted $3$D attributes (3D size, orientation, and depth) across multiple distance ranges. Compared to previous parallel prediction approaches zhang2023monodetrpu2024monodgp, MonoCoP consistently yields lower errors, particularly for distant objects, demonstrating that our MonoCoP outperforms conventional parallel prediction strategies.
• Figure 5: Qualitative Results. MonoCoP improves detection accuracy, particularly for distant objects, consistent with the results in \ref{['fig:error']}c. [Key: MonoCoP, Baseline, Ground Truth]