TLD: A Vehicle Tail Light signal Dataset and Benchmark

TL;DR

This work addresses the need for reliable interpretation of taillight signals to predict driving intentions, a capability often missing in autonomous systems due to scarce open datasets. It introduces TLD, the first large-scale taillight dataset with decoupled brake and turn signal annotations in full-frame driving scenes, plus a two-stage detection pipeline that tracks vehicles and classifies taillight states with temporal post-processing. The dataset comprises 152,690 labeled frames and about 1.6 million unlabeled frames across 17.78 hours of driving, sourced from YouTube and the LOKI dataset, including hazard-light labels to enrich the signal space. The authors provide a baseline pipeline using YOLOv10-DeepSORT for vehicle tracking and ResNet34-based taillight classifiers, offering a public benchmark to improve driving-intention understanding and ADAS safety in real-world conditions.

Abstract

Understanding other drivers' intentions is crucial for safe driving. The role of taillights in conveying these intentions is underemphasized in current autonomous driving systems. Accurately identifying taillight signals is essential for predicting vehicle behavior and preventing collisions. Open-source taillight datasets are scarce, often small and inconsistently annotated. To address this gap, we introduce a new large-scale taillight dataset called TLD. Sourced globally, our dataset covers diverse traffic scenarios. To our knowledge, TLD is the first dataset to separately annotate brake lights and turn signals in real driving scenarios. We collected 17.78 hours of driving videos from the internet. This dataset consists of 152k labeled image frames sampled at a rate of 2 Hz, along with 1.5 million unlabeled frames interspersed throughout. Additionally, we have developed a two-stage vehicle light detection model consisting of two primary modules: a vehicle detector and a taillight classifier. Initially, YOLOv10 and DeepSORT captured consecutive vehicle images over time. Subsequently, the two classifiers work simultaneously to determine the states of the brake lights and turn signals. A post-processing procedure is then used to eliminate noise caused by misidentifications and provide the taillight states of the vehicle within a given time frame. Our method shows exceptional performance on our dataset, establishing a benchmark for vehicle taillight detection. The dataset is available at https://huggingface.co/datasets/ChaiJohn/TLD/tree/main

Figures (9)

• Figure 1: Three corner cases where turn signal information is used for interaction to avoid danger. Corner case 1 and 3: The truck driver turns on the left turn signal to indicate to the following vehicle that there is an oncoming vehicle in the opposite lane, making it unsafe to overtake. When it is safe to overtake, the truck driver turns on the right turn signal to indicate that the following vehicle can overtake. Corner case 2:The truck driver uses the left turn signal to indicate that the lane ahead is narrowing or there is an obstacle.
• Figure 2: The driving scenarios in the TLD cover different times of the day (day/night), various weather conditions (sunny, rainy), and diverse settings (congested urban areas, highways, rural areas).
• Figure 3: Various lighting conditions, diverse shapes of tail lights, differing arrangements of lights between day and night, challenges with temporal integration, and insufficient resolution all contribute to making the detection of tail light status a particularly challenging task.
• Figure 4: Geographic distribution of TLD. Our dataset covers ample driving scenarios around the world.
• Figure 5: Comparison of the number and total counts of various categories between the TLD and other vehicle light datasets. The horizontal axis is presented on a logarithmic scale. Our dataset significantly surpasses the others in both the number of categories and the overall quantity.