NT-VOT211: A Large-Scale Benchmark for Night-time Visual Object Tracking

TL;DR

This paper presents NT-VOT211, a new benchmark tailored for evaluating visual object tracking algorithms in the challenging night-time conditions, and is the largest night-time tracking benchmark to-date that is specifically designed to address unique challenges such as adverse visibility, image blur, and distractors inherent to night-time tracking scenarios.

Abstract

Many current visual object tracking benchmarks such as OTB100, NfS, UAV123, LaSOT, and GOT-10K, predominantly contain day-time scenarios while the challenges posed by the night-time has been less investigated. It is primarily because of the lack of a large-scale, well-annotated night-time benchmark for rigorously evaluating tracking algorithms. To this end, this paper presents NT-VOT211, a new benchmark tailored for evaluating visual object tracking algorithms in the challenging night-time conditions. NT-VOT211 consists of 211 diverse videos, offering 211,000 well-annotated frames with 8 attributes including camera motion, deformation, fast motion, motion blur, tiny target, distractors, occlusion and out-of-view. To the best of our knowledge, it is the largest night-time tracking benchmark to-date that is specifically designed to address unique challenges such as adverse visibility, image blur, and distractors inherent to night-time tracking scenarios. Through a comprehensive analysis of results obtained from 42 diverse tracking algorithms on NT-VOT211, we uncover the strengths and limitations of these algorithms, highlighting opportunities for enhancements in visual object tracking, particularly in environments with suboptimal lighting. Besides, a leaderboard for revealing performance rankings, annotation tools, comprehensive meta-information and all the necessary code for reproducibility of results is made publicly available. We believe that our NT-VOT211 benchmark will not only be instrumental in facilitating field deployment of VOT algorithms, but will also help VOT enhancements and it will unlock new real-world tracking applications. Our dataset and other assets can be found at: {https://github.com/LiuYuML/NV-VOT211.

Figures (9)

• Figure 1: Performance comparison of 9 SOTA VOT algorithms on a dominantly day-time dataset (TrackingNet) and on our proposed night-time dataset NT-VOT211. The performance of each algorithm is significantly poor on our night-time benchmark underscoring the pressing need for night-time VOT benchmarks to enable reliable deployment of VOT algorithms in night-time scenarios.
• Figure 2: The best performing tracking algorithm is shown on each of the 12 benchmarks. AVisT focuses on adverse tracking conditions and UAVDark2021 has majorly low-light scenarios. These are also challenging scenarios for SOTA trackers. NT-VOT211 is unique for night-time content containing diverse, more challenging and extensive night-time scenes.
• Figure 3: Proposed NT-VOT211 benchmark vs. existing low-light datasets for average brightness (x-axis), brightness variance (y-axis), and the number of total annotated frames (bubble size). The proposed NT-VOT211 benchmark has lowest average intensity, highest brightness variance due to darkness of night and brightness of lights, and has the highest number of annotated frames.
• Figure 3: Original results (no finetuning) vs. finetuned res on Split Training Sequences in NT-VOT211.
• Figure 4: Sample frames from our proposed Night-Time Visual Object Tracking (NT-VOT211) dataset with ground truth bounding boxes annotated in red.