CRTrack: Low-Light Semi-Supervised Multi-object Tracking Based on Consistency Regularization

TL;DR

This work tackles pedestrian tracking in low-light conditions by introducing the LLMOT dataset and CRTrack, a semi-supervised tracking framework using consistency regularization. A teacher–student detector (YOLOX-based) learns from labeled and unlabeled nighttime data, with Consistent Adaptive Sampling Assignment to refine pseudo-labels and Adaptive Network Updating to keep the teacher aligned with the best-performing model. The association module combines appearance and motion cues with split cosine distance to maintain robust tracks under noise and blur. Experiments on LLMOT and CrowdHuman demonstrate improved detection and tracking performance over strong baselines, underscoring the value of semi-supervised learning and targeted data augmentation for real-world low-light MOT. The dataset and code are publicly available, enabling broader evaluation and development in nocturnal tracking scenarios.

Abstract

Multi-object tracking under low-light environments is prevalent in real life. Recent years have seen rapid development in the field of multi-object tracking. However, due to the lack of datasets and the high cost of annotations, multi-object tracking under low-light environments remains a persistent challenge. In this paper, we focus on multi-object tracking under low-light conditions. To address the issues of limited data and the lack of dataset, we first constructed a low-light multi-object tracking dataset (LLMOT). This dataset comprises data from MOT17 that has been enhanced for nighttime conditions as well as multiple unannotated low-light videos. Subsequently, to tackle the high annotation costs and address the issue of image quality degradation, we propose a semi-supervised multi-object tracking method based on consistency regularization named CRTrack. First, we calibrate a consistent adaptive sampling assignment to replace the static IoU-based strategy, enabling the semi-supervised tracking method to resist noisy pseudo-bounding boxes. Then, we design a adaptive semi-supervised network update method, which effectively leverages unannotated data to enhance model performance. Dataset and Code: https://github.com/ZJZhao123/CRTrack.

Figures (11)

• Figure 1: Some results of low-light tracking. Comparing feature maps of normal and low-light images shows that there are fewer features in low-light environments. This results in poor performance. Tracking models trained on normal lighting datasets perform poorly when directly applied to low-light settings, which leads to missed and false detections.
• Figure 2: The results of enhancement using the stable-diffusion model. The image on the left is the original, and the results on the right is generated by guiding the stable-diffusion model with prompts like night street, dark sky, noise, dim...
• Figure 3: The results of enhancement using CycleGAN. Through adversarial learning, low-light images are transformed into normal-light images, and normal-light images are transformed into low-light images.
• Figure 4: The results of enhancement using manually designed method. We combine multiple enhancement methods, including Gaussian blur and gamma correction, and ultimately generated the low-light enhancement result.
• Figure 5: Teacher-student network pipeline. Use the output of the teacher network as ground truth to guide the student network learning, and use the updates of the student network's parameters to fine-tune the teacher network's weights.