Enhancing Nighttime UAV Tracking with Light Distribution Suppression

TL;DR

This work tackles the challenge of nighttime UAV tracking under uneven illumination by introducing LDEnhancer, which decouples light distribution from image content via a content refinement module and a light distribution generator to produce suppression and enhancement parameter maps. A two-branch parameter inference framework and an interweave iteration adjustment mechanism enable coordinated, pixel-wise image refinement that mitigates over-enhancement of bright regions while boosting dark areas. The approach is validated on the UAVDark135 benchmark and a new NAT2024-2 dataset, showing substantial gains across multiple trackers and outperforming existing LLIE-based methods, with real-time performance on an NVIDIA Orin NX. The results suggest strong practical potential for robust nighttime UAV tracking in complex illumination conditions.

Abstract

Visual object tracking has boosted extensive intelligent applications for unmanned aerial vehicles (UAVs). However, the state-of-the-art (SOTA) enhancers for nighttime UAV tracking always neglect the uneven light distribution in low-light images, inevitably leading to excessive enhancement in scenarios with complex illumination. To address these issues, this work proposes a novel enhancer, i.e., LDEnhancer, enhancing nighttime UAV tracking with light distribution suppression. Specifically, a novel image content refinement module is developed to decompose the light distribution information and image content information in the feature space, allowing for the targeted enhancement of the image content information. Then this work designs a new light distribution generation module to capture light distribution effectively. The features with light distribution information and image content information are fed into the different parameter estimation modules, respectively, for the parameter map prediction. Finally, leveraging two parameter maps, an innovative interweave iteration adjustment is proposed for the collaborative pixel-wise adjustment of low-light images. Additionally, a challenging nighttime UAV tracking dataset with uneven light distribution, namely NAT2024-2, is constructed to provide a comprehensive evaluation, which contains 40 challenging sequences with over 74K frames in total. Experimental results on the authoritative UAV benchmarks and the proposed NAT2024-2 demonstrate that LDEnhancer outperforms other SOTA low-light enhancers for nighttime UAV tracking. Furthermore, real-world tests on a typical UAV platform with an NVIDIA Orin NX confirm the practicality and efficiency of LDEnhancer. The code is available at https://github.com/vision4robotics/LDEnhancer.

Figures (8)

• Figure 1: Overall performance of the state-of-the-art (SOTA) trackers cao2021siamapn++cao2021hiftli2019siamRPN++peng2022lpatfu2021onboard with the proposed LDEnhancer enabled or not on UAVDark135 li2022all. With the support of LDEnhancer, SOTA UAV trackers, which exhibit difficulty in nighttime scenarios with uneven light distribution, have been significantly improved. SiamAPN++ cao2021siamapn++ with LDEnhancer gains the improvement with 25.0% and 25.5% in precision and success rate, respectively. Meanwhile, LPAT peng2022lpat gets the promotion with 46.3% and 46.5% in precision and success rate, respectively.
• Figure 2: Overview of our proposed LDEnhancer. The LDEnhancer is composed of three parts, from left to right: feature extraction, light distribution-aware parameter inference, and interweave iteration enhancement.$P_S$ denotes the suppression parameter map. $P_E$ represents the enhancement parameter map.
• Figure 3: Detailed workflow of the image content refinement module.
• Figure 4: First frames of sequences from NAT2024-2. The tracking object is marked with red boxes.
• Figure 5: Overall performance of SOTA UAV trackers with LDEhancer utilized (plots with solid lines) or not (plots with dashed lines). The same color denotes the same tracker. The results demonstrate the effectiveness and superiority of the proposed methods for nighttime UAV tracking.