Boosting Visual Recognition in Real-world Degradations via Unsupervised Feature Enhancement Module with Deep Channel Prior

TL;DR

This work addresses degraded visual recognition in autonomous driving by introducing Deep Channel Prior (DCP), a prior that exploits degradation-specific channel-correlation patterns in deep features. It then proposes UFEM, a two-stage unsupervised feature enhancement module that first restores latent content and reduces artifacts, and then modulates global channel correlations guided by DCP, using losses $L_{adv}$, $L_{cyc}$, $L_{idt}$, $L_{correlation}$, and $L_{content}$. Across eight datasets and three tasks (classification, detection, segmentation), UFEM improves the performance of pre-trained models under real-world degradations, while remaining plug-and-play and data-efficient (≈100 unpaired images). This approach enhances robustness of perception systems in autonomous driving, enabling safer navigation under challenging environmental conditions, with limitations under mixed degradations and when applied to pure Transformer architectures.

Abstract

The environmental perception of autonomous vehicles in normal conditions have achieved considerable success in the past decade. However, various unfavourable conditions such as fog, low-light, and motion blur will degrade image quality and pose tremendous threats to the safety of autonomous driving. That is, when applied to degraded images, state-of-the-art visual models often suffer performance decline due to the feature content loss and artifact interference caused by statistical and structural properties disruption of captured images. To address this problem, this work proposes a novel Deep Channel Prior (DCP) for degraded visual recognition. Specifically, we observe that, in the deep representation space of pre-trained models, the channel correlations of degraded features with the same degradation type have uniform distribution even if they have different content and semantics, which can facilitate the mapping relationship learning between degraded and clear representations in high-sparsity feature space. Based on this, a novel plug-and-play Unsupervised Feature Enhancement Module (UFEM) is proposed to achieve unsupervised feature correction, where the multi-adversarial mechanism is introduced in the first stage of UFEM to achieve the latent content restoration and artifact removal in high-sparsity feature space. Then, the generated features are transferred to the second stage for global correlation modulation under the guidance of DCP to obtain high-quality and recognition-friendly features. Evaluations of three tasks and eight benchmark datasets demonstrate that our proposed method can comprehensively improve the performance of pre-trained models in real degradation conditions. The source code is available at https://github.com/liyuhang166/Deep_Channel_Prior

Figures (14)

• Figure 1: We randomly select 100 unpaired clear and hazy images from Haze-20 pei2018does and extract their shallow features using the pre-trained VGG16. Then, we calculate the average sparsity (i.e., the proportion of pixels with zero response) of each channel and find that, degradation cues increase the overall sparsity in degraded features due to the significant loss of feature content. Besides, degradation clues will lead to the introduction of artifacts, resulting in a reduction in the sparsity of some channels.
• Figure 2: The illustration of Deep Channel Prior (DCP). This illustrates that the channel correlation matrix of features is an explicit means to reflect the corruption type of degraded images, while the feature itself can not represent its degradation type. The clear, dark, and hazy images are respectively from ImageNet, ExDARK, and Haze-20. And $f$ represents the feature map of a specific channel. Similar phenomena can be observed in both VGG16 and ResNet50.
• Figure 3: The pipeline of our proposed Unsupervised Feature Enhancement Module (UFEM). The UFEM takes unpaired clear features and degraded features as input, and sequentially performs content restoration and channel correlation modulation for feature correction guided by DCP. Finally, the UFEM is seamlessly inserted into existing models to improve their performance on degraded images.
• Figure 4: The visualization comparison of enhanced features generated by the top 5 UIR methods and our UFEM, respectively.
• Figure 5: Grad-CAM attention maps comparison of our UFEM with UIR methods. We choose the Top-3 IR methods for visualization. Top-1 prediction of the classifier ResNet50 together with its confidence score is presented.