Conditional Generative Denoiser for Nighttime UAV Tracking

TL;DR

A novel conditional generative denoiser (CG-Denoiser), which breaks free from the limitations of traditional deterministic paradigms and generates the noise conditioning on the input, subsequently removing it, is proposed.

Abstract

State-of-the-art (SOTA) visual object tracking methods have significantly enhanced the autonomy of unmanned aerial vehicles (UAVs). However, in low-light conditions, the presence of irregular real noise from the environments severely degrades the performance of these SOTA methods. Moreover, existing SOTA denoising techniques often fail to meet the real-time processing requirements when deployed as plug-and-play denoisers for UAV tracking. To address this challenge, this work proposes a novel conditional generative denoiser (CGDenoiser), which breaks free from the limitations of traditional deterministic paradigms and generates the noise conditioning on the input, subsequently removing it. To better align the input dimensions and accelerate inference, a novel nested residual Transformer conditionalizer is developed. Furthermore, an innovative multi-kernel conditional refiner is designed to pertinently refine the denoised output. Extensive experiments show that CGDenoiser promotes the tracking precision of the SOTA tracker by 18.18\% on DarkTrack2021 whereas working 5.8 times faster than the second well-performed denoiser. Real-world tests with complex challenges also prove the effectiveness and practicality of CGDenoiser. Code, video demo and supplementary proof for CGDenoier are now available at: \url{https://github.com/vision4robotics/CGDenoiser}.

Figures (11)

• Figure 1: Tracking performance comparison in a typical dark scene with the proposed CGDenoiser utilized (in red) or not (in blue). The center location error (CLE) curves between predicted locations and ground truth bounding boxes are also exhibited. CGDenoiser raises tracking robustness in low-light conditions remarkably.
• Figure 2: Overall performance promotion on the DarkTrack2021 benchmark Ye2022TrackerMN for leading-edge trackers Li2019SiamRPN++EOGuo2021GraphATChen2020SiameseBAGuo2020SiamCARSFWang2019FastOO enhanced by different SOTA enhancers Li2022LearningTEYe2022TrackerMN. Symbols with the dark color represent original enhanced trackers, while the light ones represent enhanced trackers with the denoising support of CGDenoiser, which significantly improves tracking success rate and precision.
• Figure 3: The pipeline of the proposed conditional real-noise generative denoiser. Given a low-light noisy image patch, CGDenoiser dedicates to conditionally generate its noise map then remove it, resulting in dramatic improvement of UAV trackers performance.
• Figure 4: Illustration of the dual-branch posterior estimator (DPE) in the training stage. The input $c_{x}$ denotes the clean image and the output $z_{k}$ and $z_{x}$ represent posterior latent variables for kernels $k$ and real-noise map $x$ respectively. DPE consists of two posterior estimation branches, each of which encodes a clean image into the latent distribution of either kernels or noise map by feature extraction and alignment.
• Figure 5: Tracking promotion of trackers on two authoritative benchmarks Li2022AlldoYe2022TrackerMN. To further demonstrate the effectiveness of CGDenoiser, trackers with different enhancers are evaluated (the first and second column of results are tested with DCE++ Li2022LearningTE, while the last column with SCT Ye2022TrackerMN). Solid and dashed lines respectively represent the performance with and without CGDenoiser. Results with higher metrics are bolded.