Real-time Traffic Object Detection for Autonomous Driving

TL;DR

The paper tackles the challenge of real-time perception for autonomous driving by extending a lightweight pedestrian detector, LSFM, to multi-class traffic object detection while introducing a Real-Time Objective Performance (RTOP) KPI that couples accuracy with runtime. It presents the LSFM-based Efficient Traffic Object Detection framework, featuring a ConvMLP-Pin backbone and SP3-based localized feature mixing, and adapts the loss for balanced multi-class training. The study validates performance on diverse autonomous driving benchmarks, including night and adverse weather, showing that LSFM B achieves strong accuracy and LSFM P delivers superior real-time performance via RTOP, often outperforming conventional detectors. The proposed RTOP KPI provides a pragmatic metric for real-time systems, offering a meaningful balance between mAP and frame-rate and highlighting the practical impact for onboard autonomous driving perception systems.

Abstract

With recent advances in computer vision, it appears that autonomous driving will be part of modern society sooner rather than later. However, there are still a significant number of concerns to address. Although modern computer vision techniques demonstrate superior performance, they tend to prioritize accuracy over efficiency, which is a crucial aspect of real-time applications. Large object detection models typically require higher computational power, which is achieved by using more sophisticated onboard hardware. For autonomous driving, these requirements translate to increased fuel costs and, ultimately, a reduction in mileage. Further, despite their computational demands, the existing object detectors are far from being real-time. In this research, we assess the robustness of our previously proposed, highly efficient pedestrian detector LSFM on well-established autonomous driving benchmarks, including diverse weather conditions and nighttime scenes. Moreover, we extend our LSFM model for general object detection to achieve real-time object detection in traffic scenes. We evaluate its performance, low latency, and generalizability on traffic object detection datasets. Furthermore, we discuss the inadequacy of the current key performance indicator employed by object detection systems in the context of autonomous driving and propose a more suitable alternative that incorporates real-time requirements.

Figures (3)

• Figure 1: Comparison of LSFM models with different traffic object detection models on real-world autonomous driving datasets. The dotted yellow line indicates the real-time threshold. LSFM P is the only model to achieve $30 FPS$ on all datasets with reasonable $mAP$.
• Figure 2: The weighting factor $w$ of RTOP plotted against $FPS$ for different base $b$ values. Lower $b$ values increase the contribution of $p$ in RTOP, while higher values favor throughput.
• Figure 3: Qualitative comparison of LSFM B and Cascade R-CNN cai2018cascade. Car detections are indicated by the color cyan, pedestrian detections are indicated by the color red, and motorcycle detections are indicated by the color green. Other classes are ignored for simplicity in comparison. The contrast of the output images is enhanced for better visibility.