Low-Light Object Tracking: A Benchmark

TL;DR

The paper tackles the challenge of visual object tracking in low-light conditions by introducing the LLOT benchmark, a 269-sequence dataset with over 132K frames annotated across 32 object categories and 12 attributes, including a new LAI illumination metric. It evaluates 39 state-of-the-art trackers and finds substantial performance gaps under low light, prompting the development of H-DCPT, a baseline that fuses darkness cue prompts with historical prompts within a HIPTrack-derived architecture. H-DCPT demonstrates superior performance on LLOT, achieving S_AUC = 0.576, P = 0.684, and P_Norm = 0.739, and outperforms all competitors by effectively leveraging darkness-aware representation and historical context. The work emphasizes the importance of specialized benchmarks and end-to-end low-light tracking solutions for real-world deployment, providing both dataset and code to stimulate future research.

Abstract

In recent years, the field of visual tracking has made significant progress with the application of large-scale training datasets. These datasets have supported the development of sophisticated algorithms, enhancing the accuracy and stability of visual object tracking. However, most research has primarily focused on favorable illumination circumstances, neglecting the challenges of tracking in low-ligh environments. In low-light scenes, lighting may change dramatically, targets may lack distinct texture features, and in some scenarios, targets may not be directly observable. These factors can lead to a severe decline in tracking performance. To address this issue, we introduce LLOT, a benchmark specifically designed for Low-Light Object Tracking. LLOT comprises 269 challenging sequences with a total of over 132K frames, each carefully annotated with bounding boxes. This specially designed dataset aims to promote innovation and advancement in object tracking techniques for low-light conditions, addressing challenges not adequately covered by existing benchmarks. To assess the performance of existing methods on LLOT, we conducted extensive tests on 39 state-of-the-art tracking algorithms. The results highlight a considerable gap in low-light tracking performance. In response, we propose H-DCPT, a novel tracker that incorporates historical and darkness clue prompts to set a stronger baseline. H-DCPT outperformed all 39 evaluated methods in our experiments, demonstrating significant improvements. We hope that our benchmark and H-DCPT will stimulate the development of novel and accurate methods for tracking objects in low-light conditions. The LLOT and code are available at https://github.com/OpenCodeGithub/H-DCPT.

Figures (9)

• Figure 1: (a) Object tracking under favorable illumination conditions and (b) object tracking under low-light conditions. In Figure (b), the second row exhibits the results after image enhancement processing. Compared to well lighting conditions, low-light environments significantly increase the difficulty of object tracking. This is because low-light environments often present additional challenges such as high levels of noise, color distortions, lower contrast and visibility.
• Figure 2: Comparison of the average LAI (Low Ambient Illuminationg ye2022nat2021) per frame of common datasets, nighttime datasets and our LLOT dataset.
• Figure 3: Visual comparison of different low-light image enhancement methods on the LLOT dataset. The comparison demonstrates that the SCI method exhibits significant advantages when processing the LLOT dataset: (1) enhanced images show clearer object edge contours; (2) image noise is less pronounced compared to other methods; and (3) the enhancement effect more closely resembles natural human visual perception. These characteristics make SCI an ideal choice for LLOT preprocessing.
• Figure 4: Some examples of box annotations for LLOT.
• Figure 5: Attribute distribution of LLOT benchmark. The values along the matrix diagonal reflect the distribution across the entire benchmark, while each row or column depicts the joint distribution of a specific attribute subset.