Breaking Down Monocular Ambiguity: Exploiting Temporal Evolution for 3D Lane Detection

TL;DR

This work tackles the fundamental ambiguity of monocular 3D lane detection by leveraging temporal information across frames. The proposed GTA-Net introduces TGEM to enforce depth-aware geometry through a learned cost-volume between consecutive frames and TIQG to fuse temporal cues into lane queries, including a synthetic pseudo-future view to reveal distant lanes. A deformable attention-based decoder refines these queries to produce accurate 3D lane predictions, trained with a LatR-style loss that combines visibility, coordinates, category, and segmentation terms. Empirical results on the OpenLane benchmark demonstrate state-of-the-art performance across F1, category accuracy, and near/far localization, with ablations validating the importance of TGEM, TIQG, and the cost volume for robust monocular 3D lane perception.

Abstract

Monocular 3D lane detection aims to estimate the 3D position of lanes from frontal-view (FV) images. However, existing methods are fundamentally constrained by the inherent ambiguity of single-frame input, which leads to inaccurate geometric predictions and poor lane integrity, especially for distant lanes. To overcome this, we propose to unlock the rich information embedded in the temporal evolution of the scene as the vehicle moves. Our proposed Geometry-aware Temporal Aggregation Network (GTA-Net) systematically leverages the temporal information from complementary perspectives. First, Temporal Geometry Enhancement Module (TGEM) learns geometric consistency across consecutive frames, effectively recovering depth information from motion to build a reliable 3D scene representation. Second, to enhance lane integrity, Temporal Instance-aware Query Generation (TIQG) module aggregates instance cues from past and present frames. Crucially, for lanes that are ambiguous in the current view, TIQG innovatively synthesizes a pseudo future perspective to generate queries that reveal lanes which would otherwise be missed. The experiments demonstrate that GTA-Net achieves new SoTA results, significantly outperforming existing monocular 3D lane detection solutions.

Figures (5)

• Figure 1: Comparison with Monocular 3D Lane Detection Approaches. (a) Existing methods huang2023anchor3dlanebai2023curveformerguo2020genchen2022persformergarnett20193d that rely on a single frame as input are hindered by inaccuracies in 3D geometry perception and difficulties in maintaining lane integrity. (b) In contrast, our approach, which incorporates multiple frames along with synthetic future frames, facilitates improved 3D geometric perception and enables integral lane detection within the scene.
• Figure 2: Illustration of the Overall Architecture of the Proposed GTA-Net. The input images are first fed into a backbone network to extract 2D perspective features. Next, TGEM is applied to enhance the geometric perception. Simultaneously, TIQG is used to generate lane queries. The lane queries and features are sent to the decoder to obtain 3D lane predictions.
• Figure 3: An Overview of the Proposed Temporal Instance-aware Query Generation (TIQG) Module. TIQG strategically integrates temporal cues into the query generation process, thereby enhancing its capacity to exploit temporal instance information for promoting lane integrity.
• Figure 4: Monocular Lane Detection Results of Setting Current and the Synthetic Future Frames as Input. The white line represents the predicted lane line and the red line denotes the gt. We see that the lane line is missing in the predictions when accepting the current frame. In contrast, there is a white line in the visual results for treating the synthetic future frame as input.
• Figure 5: Illustration of Predicted 3D Lanes on the OpenLane Dataset. We present the predicted 3D lanes in both the perspective view and the 3D spatial representation. The red lines denote the ground truth lanes, while the other colored lines represent the predicted lanes.