RAUCA: A Novel Physical Adversarial Attack on Vehicle Detectors via Robust and Accurate Camouflage Generation

TL;DR

The core of RAUCA is a novel neural rendering component, Neural Renderer Plus (NRP), which can accurately project vehicle textures and render images with environmental characteristics such as lighting and weather, and integrate a multi-weather dataset for camouflage generation.

Abstract

Adversarial camouflage is a widely used physical attack against vehicle detectors for its superiority in multi-view attack performance. One promising approach involves using differentiable neural renderers to facilitate adversarial camouflage optimization through gradient back-propagation. However, existing methods often struggle to capture environmental characteristics during the rendering process or produce adversarial textures that can precisely map to the target vehicle, resulting in suboptimal attack performance. Moreover, these approaches neglect diverse weather conditions, reducing the efficacy of generated camouflage across varying weather scenarios. To tackle these challenges, we propose a robust and accurate camouflage generation method, namely RAUCA. The core of RAUCA is a novel neural rendering component, Neural Renderer Plus (NRP), which can accurately project vehicle textures and render images with environmental characteristics such as lighting and weather. In addition, we integrate a multi-weather dataset for camouflage generation, leveraging the NRP to enhance the attack robustness. Experimental results on six popular object detectors show that RAUCA consistently outperforms existing methods in both simulation and real-world settings.

Figures (7)

• Figure 1: Comparison of different adversarial camouflage under sunny (first row) and foggy (second row) environments, where only our method succeeds in both cases. (a) A car with normal texture; (b) DAS wang2021dual. (c) and (d) are top-performed methods FCA wang2022fca and ACTIVE Suryanto_2023_ICCV, respectively. (e) Our method RAUCA.
• Figure 2: The overview of RAUCA. First, a multi-weather dataset is created using CARLA, which includes car images, corresponding mask images, and camera angles. Then the car images are segmented using the mask images to obtain the foreground car and background images. The foreground car, together with the 3D model and the camera angle is passed through the NRP rendering component for rendering. The rendered image is then seamlessly integrated with the background. Finally, we optimize the adversarial camouflage through back-propagation with our devised loss function computed from the output of the object detector.
• Figure 3: Comparison of rendering results of neural renderers used in different methods. The first row shows the results obtained by different neural renderers (already blended with the background) and the second row shows the rendered results in UE4. Our renderer is the only one that does both foreground environment rendering and texture rendering similar to UE4.
• Figure 4: Attack comparison at night, a weather condition that has been included in our training set.
• Figure 5: Attack comparison in the rainy day, a weather condition that hasn't appeared in our training set.