Sketch and Refine: Towards Fast and Accurate Lane Detection

TL;DR

The paper addresses the challenge of real-time lane detection under diverse real-world conditions. It introduces the Sketch-and-Refine SRLane framework, which quickly sketches lanes by predicting a local direction map to generate lane proposals and then refines them with a Lane Segment Association Module and adaptive multi-level feature sampling. Key contributions include the direction-based lane sketch, the LSAM mechanism with bipartite supervision, and a lightweight multi-scale sampling strategy that yields high speed (278 FPS) and strong accuracy (F1 78.9% on CULane). The approach provides a practical, end-to-end lane detector that is suitable for resource-constrained autonomous driving systems and serves as a strong baseline for future real-time lane detection work.

Abstract

Lane detection is to determine the precise location and shape of lanes on the road. Despite efforts made by current methods, it remains a challenging task due to the complexity of real-world scenarios. Existing approaches, whether proposal-based or keypoint-based, suffer from depicting lanes effectively and efficiently. Proposal-based methods detect lanes by distinguishing and regressing a collection of proposals in a streamlined top-down way, yet lack sufficient flexibility in lane representation. Keypoint-based methods, on the other hand, construct lanes flexibly from local descriptors, which typically entail complicated post-processing. In this paper, we present a "Sketch-and-Refine" paradigm that utilizes the merits of both keypoint-based and proposal-based methods. The motivation is that local directions of lanes are semantically simple and clear. At the "Sketch" stage, local directions of keypoints can be easily estimated by fast convolutional layers. Then we can build a set of lane proposals accordingly with moderate accuracy. At the "Refine" stage, we further optimize these proposals via a novel Lane Segment Association Module (LSAM), which allows adaptive lane segment adjustment. Last but not least, we propose multi-level feature integration to enrich lane feature representations more efficiently. Based on the proposed "Sketch and Refine" paradigm, we propose a fast yet effective lane detector dubbed "SRLane". Experiments show that our SRLane can run at a fast speed (i.e., 278 FPS) while yielding an F1 score of 78.9\%. The source code is available at: https://github.com/passerer/SRLane.

Figures (7)

• Figure 1: Illustrations of lane detection methods. (a) An example of keypoint-based methods, where local keypoints are grouped based on their offsets (orange colored arrow) from each other to reconstruct the whole lane. (b) An example of anchor-based methods which use pre-defined line anchors (cyan colored line) to match and predict lanes. (c) The proposed paradigm first sketches local directions for a set of keypoints. Then it extends the estimated local directions to build lane proposals. (d) Refinement of lane segments via the proposed Segment Association Module (i.e., the dashed line segment is replaced to better fit the ground truth).
• Figure 2: Overall pipeline of SRLane, which can be decomposed into two stages: lane sketch and refinement. In the sketch stage, the last feature map from backbone is encoded to create a local direction map, where a set of lane proposals are initialized. In the refinement stage, features of proposals are adaptively sampled from multi-scale feature maps and then enhanced by the Lane Segment Association Module. After that, they are fed into the classification and regression branch to produce final results.
• Figure 3: Illustration of multi-level feature integration. Before sampling, all feature maps are first transformed by a single convolutional layer to be of the same channel dimension $d$. $N_p$ denotes the number of sample points. The sampled features are finally projected to a 1D feature vector with channel $c$.
• Figure 4: Illustration of Lane Segment Association Module. $G$ denotes the number of groups and $L$ is the number of lane proposals. FFN is the Feed Forward Network detr. For simplicity, the batch dimension is ignored.
• Figure 5: Performance comparison with different number of proposals.