Exploiting Polarized Material Cues for Robust Car Detection

TL;DR

This work tackles robust car detection under adverse lighting and dense scenes by introducing polarization cues—specifically trichromatic AoLP and DoLP—alongside RGB information. It presents PCDNet, a multimodal network with three modules: Polarization Integration, Material Perception, and Cross Domain Demand Query, and a pixel-aligned RGBP-Car dataset to enable learning of polarization-based material cues. Empirical results show PCDNet outperforms state-of-the-art detectors, especially in challenging conditions, demonstrating that polarization cues provide discriminative material properties for reliable detection. The approach offers practical impact for safer automated driving by enhancing perception in real-world, non-ideal imaging scenarios and providing a new dataset to foster polarization-based vision research.

Abstract

Car detection is an important task that serves as a crucial prerequisite for many automated driving functions. The large variations in lighting/weather conditions and vehicle densities of the scenes pose significant challenges to existing car detection algorithms to meet the highly accurate perception demand for safety, due to the unstable/limited color information, which impedes the extraction of meaningful/discriminative features of cars. In this work, we present a novel learning-based car detection method that leverages trichromatic linear polarization as an additional cue to disambiguate such challenging cases. A key observation is that polarization, characteristic of the light wave, can robustly describe intrinsic physical properties of the scene objects in various imaging conditions and is strongly linked to the nature of materials for cars (e.g., metal and glass) and their surrounding environment (e.g., soil and trees), thereby providing reliable and discriminative features for robust car detection in challenging scenes. To exploit polarization cues, we first construct a pixel-aligned RGB-Polarization car detection dataset, which we subsequently employ to train a novel multimodal fusion network. Our car detection network dynamically integrates RGB and polarization features in a request-and-complement manner and can explore the intrinsic material properties of cars across all learning samples. We extensively validate our method and demonstrate that it outperforms state-of-the-art detection methods. Experimental results show that polarization is a powerful cue for car detection.

Figures (7)

• Figure 1: Car detections (indicated by green bounding boxes) obtained with the RGB-only methods of Deformable DETR zhu2020deformable and YOLOv7 wang2022yolov7 compared to our RGB-Polarization Car Detection Network (PCDNet). Both prior methods fail to distinguish mirrored cars from real ones due to the similar visual appearances. In contrast, our method can handle such ambiguity and correctly detect the real car in scenes with the help of intrinsic material properties revealed by the polarization cues.
• Figure 2: Overview of PCDNet and its three main modules: the Polarization Integration (PI) module, the Material Spatial/Channel Perception (MSP/MCP) module, and the Cross Domain Demand Query (CDDQ) module.
• Figure 3: RGBP-Car Examples. The first column displays the RGB intensity (top) and the corresponding annotation (bottom). The next three columns show the AoLP (top) and DoLP (bottom) measurements for the red, green, and blue channels, respectively. From top to bottom are scenes of stopped cars in a rainy parking lot, dense cars in an outdoor parking lot, and driving cars on a clear night road, respectively. (The low-light RGB image is enhanced by ZeroDCE guo2020zero (with orange frame) for visualization.)
• Figure 4: The images in our RGB-P Car dataset vary in terms of (a) scenarios and (b) the number of car instances.
• Figure 5: Qualitative comparison of PCDNet against state-of-the-art detectors retrained on RGB-P Car dataset.