Learnability-Driven Submodular Optimization for Active Roadside 3D Detection

TL;DR

This work addresses the challenge of active learning for roadside monocular 3D detection, where cross-view or cross-modal signals are often unavailable and many scenes contain inherently ambiguous objects. It introduces LH3D, a learnability-driven, three-stage submodular active learning framework that selects depth-confident, semantically balanced, and geometrically varied scenes by optimizing concave-over-modular objectives, enabling efficient labeling with limited budgets. The method demonstrates that prioritizing learnability over uncertainty yields stronger supervision, achieving substantial portions of fully supervised performance on DAIR-V2X-I and Rope3D with only a fraction of annotations. Theoretical guarantees from submodular optimization support the greedy approach, and extensive experiments, ablations, and a human study validate the practical impact of focusing on learnable samples for roadside BEV perception.

Abstract

Roadside perception datasets are typically constructed via cooperative labeling between synchronized vehicle and roadside frame pairs. However, real deployment often requires annotation of roadside-only data due to hardware and privacy constraints. Even human experts struggle to produce accurate labels without vehicle-side data (image, LIDAR), which not only increases annotation difficulty and cost, but also reveals a fundamental learnability problem: many roadside-only scenes contain distant, blurred, or occluded objects whose 3D properties are ambiguous from a single view and can only be reliably annotated by cross-checking paired vehicle--roadside frames. We refer to such cases as inherently ambiguous samples. To reduce wasted annotation effort on inherently ambiguous samples while still obtaining high-performing models, we turn to active learning. This work focuses on active learning for roadside monocular 3D object detection and proposes a learnability-driven framework that selects scenes which are both informative and reliably labelable, suppressing inherently ambiguous samples while ensuring coverage. Experiments demonstrate that our method, LH3D, achieves 86.06%, 67.32%, and 78.67% of full-performance for vehicles, pedestrians, and cyclists respectively, using only 25% of the annotation budget on DAIR-V2X-I, significantly outperforming uncertainty-based baselines. This confirms that learnability, not uncertainty, matters for roadside 3D perception.

Figures (8)

• Figure 1: Human study: learnable vs. ambiguous samples. Images are categorized as learnable or ambiguous based on how difficult they are to interpret from a single monocular view. Using this partition (while training only with the dataset’s original ground-truth labels), detectors trained on the ambiguous split achieve lower AP on cars and pedestrians under the same annotation budget and class balance, with cyclists remaining similar. This indicates that ambiguous samples provide weaker monocular supervision.
• Figure 2: Left: Our learnability-driven active learning pipeline for roadside BEV 3D detection. Right: The proposed LH3D three-stage selector—depth confidence, semantic balance, and geometric variation— which selects images that are both reliably learnable and informative for monocular roadside perception.
• Figure 3: Global Class Diversity Entropy across AL rounds. LH3D consistently achieves higher entropy than baselines, showing more balanced sampling of Car, Pedestrian, and Cyclist and preventing the increasingly imbalanced selections observed in other methods.
• Figure 4: Active learning performance progression on the DAIR-V2X-I validation set using the BEVHeight backbone under the hard modes.
• Figure 5: Visulization results of baselines and our proposed method. Our method (LH3D) successfully detects the pedestrian that other active learning baselines fail to identify in complex traffic environments. The 3D bounding boxes for vehicles, pedestrians, and cyclists are shown in green, blue, and red, respectively.