SparseLaneSTP: Leveraging Spatio-Temporal Priors with Sparse Transformers for 3D Lane Detection

TL;DR

SparseLaneSTP tackles 3D lane detection by fusing lane-aware geometric priors with a sparse transformer and incorporating temporal information through a memory-based, ego-motion-aware fusion strategy. It introduces a continuous Catmull-Rom spline lane representation, a spatio-temporal attention mechanism with dedicated masks for intra-lane, inter-lane, and historical relations, and regularization to promote smooth, consistent predictions. The approach is complemented by an auto-labeling pipeline that yields accurate long-range 3D lane labels up to 250 m, including occlusion information, enabling robust training. Experimental results on OpenLane, ONCE-3DLanes, and a new dataset show state-of-the-art performance and stronger robustness in challenging visibility and long-range scenarios, highlighting practical benefits for autonomous driving systems.

Abstract

3D lane detection has emerged as a critical challenge in autonomous driving, encompassing identification and localization of lane markings and the 3D road surface. Conventional 3D methods detect lanes from dense birds-eye-viewed (BEV) features, though erroneous transformations often result in a poor feature representation misaligned with the true 3D road surface. While recent sparse lane detectors have surpassed dense BEV approaches, they completely disregard valuable lane-specific priors. Furthermore, existing methods fail to utilize historic lane observations, which yield the potential to resolve ambiguities in situations of poor visibility. To address these challenges, we present SparseLaneSTP, a novel method that integrates both geometric properties of the lane structure and temporal information into a sparse lane transformer. It introduces a new lane-specific spatio-temporal attention mechanism, a continuous lane representation tailored for sparse architectures as well as temporal regularization. Identifying weaknesses of existing 3D lane datasets, we also introduce a precise and consistent 3D lane dataset using a simple yet effective auto-labeling strategy. Our experimental section proves the benefits of our contributions and demonstrates state-of-the-art performance across all detection and error metrics on existing 3D lane detection benchmarks as well as on our novel dataset.

Figures (14)

• Figure 1: Overview of our method SparseLaneSTP. Sparse lane queries are processed by a transformer integrating temporal knowledge and spatial lane structure priors in a novel spatio-temporal attention. Based on these priors, we formulate spatial and temporally consistent regularization. Finally, our network predicts control points defining our new continuous 3D lane representation.
• Figure 2: CR spline and B-Spline control points in comparison, both fit to the ground truth. B-Spline control points do not align with the curve and are therefore not suiting the sparse query design, whereas CR control points exactly match curve geometry.
• Figure 3: Overview of our spatio-temporal attention module.
• Figure 4: Our auto-labeling strategy.
• Figure 5: Examples from OpenLane compared to ours.