Thermal Detection of People with Mobility Restrictions for Barrier Reduction at Traffic Lights Controlled Intersections

TL;DR

This work tackles privacy and robustness challenges in RGB-based pedestrian detection at signal-controlled intersections by introducing a thermal-camera, detector-based system for barrier-free crossings. It introduces TD4PWMR, a thermal infrared dataset for people with mobility restrictions, and YOLO-Thermal, a YOLOv8-based detector augmented with Triplet-Attention, SPD-Conv, SPPFCSPC, and Quality Focal Loss to improve thermal-pedestrian detection. The detector informs a smart traffic-light controller that dynamically extends green phases and amplifies audible guidance, with priority given to visually impaired pedestrians; maximum green-time extensions are $T^{max}_{g,ext}=6\,s$ for walking impairments, $T^{max}_{g,ext}=8\,s$ for visual impairments, and $T^{max}_{g,ext}=3\,s$ for mobility burden. Experiments on TD4PWMR demonstrate state-of-the-art accuracy and real-time performance, confirming a privacy-preserving, robust approach to barrier-free intersections with practical deployment potential.

Abstract

Rapid advances in deep learning for computer vision have driven the adoption of RGB camera-based adaptive traffic light systems to improve traffic safety and pedestrian comfort. However, these systems often overlook the needs of people with mobility restrictions. Moreover, the use of RGB cameras presents significant challenges, including limited detection performance under adverse weather or low-visibility conditions, as well as heightened privacy concerns. To address these issues, we propose a fully automated, thermal detector-based traffic light system that dynamically adjusts signal durations for individuals with walking impairments or mobility burden and triggers the auditory signal for visually impaired individuals, thereby advancing towards barrier-free intersection for all users. To this end, we build the thermal dataset for people with mobility restrictions (TD4PWMR), designed to capture diverse pedestrian scenarios, particularly focusing on individuals with mobility aids or mobility burden under varying environmental conditions, such as different lighting, weather, and crowded urban settings. While thermal imaging offers advantages in terms of privacy and robustness to adverse conditions, it also introduces inherent hurdles for object detection due to its lack of color and fine texture details and generally lower resolution of thermal images. To overcome these limitations, we develop YOLO-Thermal, a novel variant of the YOLO architecture that integrates advanced feature extraction and attention mechanisms for enhanced detection accuracy and robustness in thermal imaging. Experiments demonstrate that the proposed thermal detector outperforms existing detectors, while the proposed traffic light system effectively enhances barrier-free intersection. The source codes and dataset are available at https://github.com/leon2014dresden/YOLO-THERMAL.

Figures (3)

• Figure 1: One representative example from each class in the proposed TD4PWMR dataset.
• Figure 2: Class distribution, highlighting the significant imbalance in the frequency of different classes.
• Figure 3: Visualization of raw images (top) and corresponding LayerCAM heatmaps (bottom). Warmer colors indicate regions of higher activation.