Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TLD-READY: Traffic Light Detection -- Relevance Estimation and Deployment Analysis

Nikolai Polley, Svetlana Pavlitska, Yacin Boualili, Patrick Rohrbeck, Paul Stiller, Ashok Kumar Bangaru, J. Marius Zöllner

TL;DR

This work tackles robust, real-time camera-based traffic light detection and map-free relevance estimation for autonomous driving. It benchmark-tests three modern detectors across four datasets, and introduces a novel lane-marking–driven relevance estimation pipeline that does not rely on prior HD-maps. The approach achieves strong real-world performance, including a 52 ms end-to-end latency on a real vehicle and substantial improvement in relevance estimation through directional arrows; however, generalization to unseen domains remains challenging and may require fine-tuning. By open-sourcing models, data labels for arrow relevance, and code, the paper provides a practical foundation for further research and deployment in diverse driving scenarios.

Abstract

Effective traffic light detection is a critical component of the perception stack in autonomous vehicles. This work introduces a novel deep-learning detection system while addressing the challenges of previous work. Utilizing a comprehensive dataset amalgamation, including the Bosch Small Traffic Lights Dataset, LISA, the DriveU Traffic Light Dataset, and a proprietary dataset from Karlsruhe, we ensure a robust evaluation across varied scenarios. Furthermore, we propose a relevance estimation system that innovatively uses directional arrow markings on the road, eliminating the need for prior map creation. On the DriveU dataset, this approach results in 96% accuracy in relevance estimation. Finally, a real-world evaluation is performed to evaluate the deployment and generalizing abilities of these models. For reproducibility and to facilitate further research, we provide the model weights and code: https://github.com/KASTEL-MobilityLab/traffic-light-detection.

TLD-READY: Traffic Light Detection -- Relevance Estimation and Deployment Analysis

TL;DR

This work tackles robust, real-time camera-based traffic light detection and map-free relevance estimation for autonomous driving. It benchmark-tests three modern detectors across four datasets, and introduces a novel lane-marking–driven relevance estimation pipeline that does not rely on prior HD-maps. The approach achieves strong real-world performance, including a 52 ms end-to-end latency on a real vehicle and substantial improvement in relevance estimation through directional arrows; however, generalization to unseen domains remains challenging and may require fine-tuning. By open-sourcing models, data labels for arrow relevance, and code, the paper provides a practical foundation for further research and deployment in diverse driving scenarios.

Abstract

Effective traffic light detection is a critical component of the perception stack in autonomous vehicles. This work introduces a novel deep-learning detection system while addressing the challenges of previous work. Utilizing a comprehensive dataset amalgamation, including the Bosch Small Traffic Lights Dataset, LISA, the DriveU Traffic Light Dataset, and a proprietary dataset from Karlsruhe, we ensure a robust evaluation across varied scenarios. Furthermore, we propose a relevance estimation system that innovatively uses directional arrow markings on the road, eliminating the need for prior map creation. On the DriveU dataset, this approach results in 96% accuracy in relevance estimation. Finally, a real-world evaluation is performed to evaluate the deployment and generalizing abilities of these models. For reproducibility and to facilitate further research, we provide the model weights and code: https://github.com/KASTEL-MobilityLab/traffic-light-detection.
Paper Structure (11 sections, 4 figures, 5 tables)

This paper contains 11 sections, 4 figures, 5 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Traffic Light Detection
  4. Relevance Estimation
  5. DATASETS AND LABELS
  6. MODEL EVALUATION ON VARIETY OF DATASETS
  7. Training Setup for Detection Task
  8. Evaluation of Detection Performance
  9. RELEVANCE ESTIMATION
  10. REAL-WORLD EVALUATION
  11. CONCLUSION

Figures (4)

  • Figure 1: DTLD-YOLOv8x model during a test drive in Karlsruhe. For clarity, close-ups of traffic lights are added, and icons are used to visualize the predictions of the model.
  • Figure 2: Since the DTLD dataset does not include images of the class arrow_straight_right red-yellow or the pictogram arrow_straight_left, it erroneously classifies the pictograms as circle and arrow_straight.
  • Figure 3: Detection of arrow markings and traffic lights on DTLD dataset. Relevant directional arrows are marked in green. All traffic lights matched to these relevant arrows are also marked in green. Non-relevant arrows and traffic lights are kept in white. The rightmost traffic light is relevant, as in this situation, the circle pictogram encompasses the straight arrow
  • Figure 4: Research vehicle CoCar NextGen used for test drives.