Automatic Labelling for Low-Light Pedestrian Detection

TL;DR

The paper tackles low-light pedestrian detection by introducing an automated infrared-to-RGB labeling pipeline. It trains IR detectors on daytime IR data, transfers the detections to aligned RGB frames to create autolabels, and then trains RGB detectors for low-light conditions using these labels. Across multiple architectures, autolabel-trained RGB detectors frequently outperform those trained with ground-truth labels, with DETR plus a 0.25 confidence threshold delivering the strongest overall performance. This approach reduces the need for manual labeling and suggests a scalable path to improve safety-critical pedestrian detection in challenging lighting, while highlighting the importance of model choice and sensor calibration. Future work includes expanding datasets, testing other infrared models, and extending the strategy to related tasks like pose estimation.

Abstract

Pedestrian detection in RGB images is a key task in pedestrian safety, as the most common sensor in autonomous vehicles and advanced driver assistance systems is the RGB camera. A challenge in RGB pedestrian detection, that does not appear to have large public datasets, is low-light conditions. As a solution, in this research, we propose an automated infrared-RGB labeling pipeline. The proposed pipeline consists of 1) Infrared detection, where a fine-tuned model for infrared pedestrian detection is used 2) Label transfer process from the infrared detections to their RGB counterparts 3) Training object detection models using the generated labels for low-light RGB pedestrian detection. The research was performed using the KAIST dataset. For the evaluation, object detection models were trained on the generated autolabels and ground truth labels. When compared on a previously unseen image sequence, the results showed that the models trained on generated labels outperformed the ones trained on ground-truth labels in 6 out of 9 cases for the mAP@50 and mAP@50-95 metrics. The source code for this research is available at https://github.com/BouzoulasDimitrios/IR-RGB-Automated-LowLight-Pedestrian-Labeling

Figures (5)

• Figure 1: Autolabelling pipeline, using unlabelled low-light IR-RGB image pairs as an input, the system outputs labelled RGB images.
• Figure 2: Precision-Recall curves for IR models used for autolabelling.
• Figure 3: Precision-Recall (PR) curves for the YOLO, RCNN, and DETR RGB detection models when trained on autolabels generated with the IR models versus ground truth labels.
• Figure 4: Examples of predictions made with different prediction model - training label combinations. Columns 1-2 correspond to the models trained with our autolabels, column 3 to the models trained with ground truth labels and columns 4 and 5 are the ground truth in infrared and RGB.
• Figure 5: KAIST ground truth difficult case labels, where pedestrians are poorly visible in the RGB frame.