Physical 3D Adversarial Attacks against Monocular Depth Estimation in Autonomous Driving

TL;DR

3D Depth Fool (3D2 Fool), the first 3D texture-based adversarial attack against MDE models, is proposed, specifically optimized to generate 3D adversarial textures agnostic to model types of vehicles and to have improved robustness in bad weather conditions, such as rain and fog.

Abstract

Deep learning-based monocular depth estimation (MDE), extensively applied in autonomous driving, is known to be vulnerable to adversarial attacks. Previous physical attacks against MDE models rely on 2D adversarial patches, so they only affect a small, localized region in the MDE map but fail under various viewpoints. To address these limitations, we propose 3D Depth Fool (3D$^2$Fool), the first 3D texture-based adversarial attack against MDE models. 3D$^2$Fool is specifically optimized to generate 3D adversarial textures agnostic to model types of vehicles and to have improved robustness in bad weather conditions, such as rain and fog. Experimental results validate the superior performance of our 3D$^2$Fool across various scenarios, including vehicles, MDE models, weather conditions, and viewpoints. Real-world experiments with printed 3D textures on physical vehicle models further demonstrate that our 3D$^2$Fool can cause an MDE error of over 10 meters.

Figures (10)

• Figure 1: (a) Existing 2D adversarial patch-based attacks APARATESPOO and (b) their modified versions with 3D adversarial textures fail to completely remove the vehicle from the MDE map, while (c) our 3D$^2$Fool with robust 3D adversarial textures makes the car vanish.
• Figure 2: Overview of our 3D$^2$Fool attack against MDE models. 3D$^2$Fool optimizes the adversarial texture seed $t_s$ via backpropagation using $L_{\mathrm{total}}$ through our new texture conversion (TC) and physical augmentation (PA) modules.
• Figure 3: The initial texture seed is transferred into a specified-size texture through the texture conversion module.
• Figure 4: Comparison of our attack and other attacks in Carla simulation. The first column shows the normal vehicle and the rest columns show the vehicles covered with adversarial textures achieved by different attacks. The depth estimation map is generated by Monodepth2.
• Figure 5: Attack comparison under different weather conditions.