The ATLAS of Traffic Lights: A Reliable Perception Framework for Autonomous Driving

TL;DR

This work tackles the reliability of camera-based traffic light perception for autonomous driving in urban environments by introducing the ATLAS dataset, a multi-camera collection with extensive state and pictogram annotations across varied weather, and by presenting a modular perception framework. The framework combines state-of-the-art detectors, a projection-and-association pipeline built on a minimum-cost matching strategy, and a circular-buffer decision module to stabilize state estimates for traffic-light signals. Key contributions include (1) the ATLAS dataset with 33{,}044 images and 72{,}998 bounding boxes across 25 pictogram-state classes, (2) improved detectors trained on ATLAS with better generalization, (3) a robust mapping between image detections and HD-map references using a Hungarian-based assignment, and (4) real-world deployment showing low end-to-end latency (~184 ms) and high reliability for decision-making at intersections. Collectively, these advances enhance the safety and reliability of autonomous driving at traffic-light-controlled intersections under diverse conditions.

Abstract

Traffic light perception is an essential component of the camera-based perception system for autonomous vehicles, enabling accurate detection and interpretation of traffic lights to ensure safe navigation through complex urban environments. In this work, we propose a modularized perception framework that integrates state-of-the-art detection models with a novel real-time association and decision framework, enabling seamless deployment into an autonomous driving stack. To address the limitations of existing public datasets, we introduce the ATLAS dataset, which provides comprehensive annotations of traffic light states and pictograms across diverse environmental conditions and camera setups. This dataset is publicly available at https://url.fzi.de/ATLAS. We train and evaluate several state-of-the-art traffic light detection architectures on ATLAS, demonstrating significant performance improvements in both accuracy and robustness. Finally, we evaluate the framework in real-world scenarios by deploying it in an autonomous vehicle to make decisions at traffic light-controlled intersections, highlighting its reliability and effectiveness for real-time operation.

Figures (6)

• Figure 1: Annotated images from ATLAS. The top image captures a scene recorded in rain, featuring a traffic light in the red-yellow state. The bottom left image shows multiple red traffic lights with distinct pictograms differentiated by color. The bottom right image demonstrates the necessity of a wide FOV, as the on-demand traffic light is not visible within a standard camera's field of view.
• Figure 2: We start annotating traffic lights with two-pixel width. Most annotations are four pixels wide.
• Figure 3: ATLAS contains twenty-five unique pictogram-state classes. As expected, the class imbalance is quite large.
• Figure 4: Overview of our proposed traffic light perception framework. An autonomous vehicle approaches an intersection intending to make a right turn. Two detectors identify traffic lights in both camera streams. The detections are projected and associated with traffic light reference positions from the HD map using the Hungarian algorithm. Associated detections are stored in their respective circular buffers, and the system reasons over all traffic lights in a signal group to determine the final state. In this example, despite an incorrect state and pictogram prediction by the detector, the decision module correctly infers the true traffic light state.
• Figure 5: CoCar NextGen, positioned in a right-turn lane, experiences a localization error that shifts the 3D projected traffic light positions (spheres) rightward. Despite this, our framework effectively minimizes the overall cost and correctly associates the red traffic light, as indicated by the blue and teal bounding boxes. The blue bounding box encompasses the relevant detection for the ego vehicle, and the teal bounding box shows high confidence. At the top, the decision for the autonomous driving stack and its color-coded confidence are displayed.