PedDet: Adaptive Spectral Optimization for Multimodal Pedestrian Detection

TL;DR

PedDet addresses two core challenges in multimodal pedestrian detection: insufficient fusion of RGB and infrared information and sensitivity to illumination changes. It introduces the Multi-scale Spectral Feature Perception Module (MSFPM) for adaptive spectral fusion and the Illumination Robustness Feature Decoupling Module (IRFDM) to separate pedestrian features from background under varying lighting, complemented by a contrastive learning paradigm to sharpen intermodal discrimination. The approach is built on a YOLOv10-based backbone with parallel RGB/IR branches and a joint loss that integrates classification, localization, confidence, decoupling, and contrastive objectives. Empirical evaluation on LLVIP and MSRS demonstrates state-of-the-art performance and improved robustness in low-light and challenging environments, marking a meaningful advance for safe and reliable pedestrian detection in intelligent transportation systems. Code will be released at the provided repository.

Abstract

Pedestrian detection in intelligent transportation systems has made significant progress but faces two critical challenges: (1) insufficient fusion of complementary information between visible and infrared spectra, particularly in complex scenarios, and (2) sensitivity to illumination changes, such as low-light or overexposed conditions, leading to degraded performance. To address these issues, we propose PedDet, an adaptive spectral optimization complementarity framework specifically enhanced and optimized for multispectral pedestrian detection. PedDet introduces the Multi-scale Spectral Feature Perception Module (MSFPM) to adaptively fuse visible and infrared features, enhancing robustness and flexibility in feature extraction. Additionally, the Illumination Robustness Feature Decoupling Module (IRFDM) improves detection stability under varying lighting by decoupling pedestrian and background features. We further design a contrastive alignment to enhance intermodal feature discrimination. Experiments on LLVIP and MSDS datasets demonstrate that PedDet achieves state-of-the-art performance, improving the mAP by 6.6% with superior detection accuracy even in low-light conditions, marking a significant step forward for road safety. Code will be available at https://github.com/AIGeeksGroup/PedDet.

Figures (10)

• Figure 1: Modules A and B illustrate the distinctions between the existing pedestrian detection frameworks and our proposed approach. Methods in Module A identify and locate targets only by fusing features from multiple levels. Module B shows that our method takes advantage of the Multi-scale Spectral Feature Perception Module (MSFPM) and the Illumination Robustness Feature Decoupling Module (IRFDM) to improve detection performance
• Figure 1: Training curves of LLVIP dataset.
• Figure 2: The overall pipeline of PedDet.
• Figure 3: The Architecture of Feature Extractor.
• Figure 3: Visualization of PedDet results.