Feature Corrective Transfer Learning: End-to-End Solutions to Object Detection in Non-Ideal Visual Conditions

TL;DR

The paper tackles robust object detection under non-ideal imaging conditions by proposing Feature Corrective Transfer Learning (FCTL), an end-to-end framework that avoids traditional image restoration steps. It builds a Non-Ideal Image Transfer Fast-RCNN (NITF-RCNN) with a static backbone from ideal RGB data and a dynamic non-ideal backbone, trained with a joint detection loss and the Extended Area Novel Structural Discrepancy Loss (EANSDL) to align non-ideal feature maps to their ideal counterparts. The core novelty lies in EANSDL, which combines local gradient differences and extended-area gradient consistency, modulated by a time-varying attenuation factor, to supervise feature-map correction. Empirically, FCTL yields consistent mAP gains across Rainy-, Foggy-, Dark-, and RAW-KITTI variants, approaching ideal performance on Raw-KITTI and highlighting the method’s potential to generalize to other domains where imaging conditions degrade features, all without ISP-based image restoration.

Abstract

A significant challenge in the field of object detection lies in the system's performance under non-ideal imaging conditions, such as rain, fog, low illumination, or raw Bayer images that lack ISP processing. Our study introduces "Feature Corrective Transfer Learning", a novel approach that leverages transfer learning and a bespoke loss function to facilitate the end-to-end detection of objects in these challenging scenarios without the need to convert non-ideal images into their RGB counterparts. In our methodology, we initially train a comprehensive model on a pristine RGB image dataset. Subsequently, non-ideal images are processed by comparing their feature maps against those from the initial ideal RGB model. This comparison employs the Extended Area Novel Structural Discrepancy Loss (EANSDL), a novel loss function designed to quantify similarities and integrate them into the detection loss. This approach refines the model's ability to perform object detection across varying conditions through direct feature map correction, encapsulating the essence of Feature Corrective Transfer Learning. Experimental validation on variants of the KITTI dataset demonstrates a significant improvement in mean Average Precision (mAP), resulting in a 3.8-8.1% relative enhancement in detection under non-ideal conditions compared to the baseline model, and a less marginal performance difference within 1.3% of the mAP@[0.5:0.95] achieved under ideal conditions by the standard Faster RCNN algorithm.

Figures (2)

• Figure 1: Architecture of Non-Ideal Image Transfer Faster RCNN (NITF-RCNN) Model
• Figure 2: Detection Results of NITF-RCNN on Derivative Images of ID 000332 from the KITTI Dataset, where (a) represents the original image from the KITTI dataset detected using the Faster RCNN algorithm for comparison.