Projection-based Adversarial Attack using Physics-in-the-Loop Optimization for Monocular Depth Estimation

TL;DR

This work addresses the vulnerability of monocular depth estimation (MDE) to physical adversarial perturbations by introducing a projection-based attack that casts perturbation light onto target objects. It employs physics-in-the-loop (PITL) optimization combined with a black-box Separable Covariance Matrix Adaptation Evolution Strategy (sep-CMA-ES) to design RGB light patterns that minimize the depth-estimation error relative to a target depth map, evaluated in real environments. The approach formalizes the objective $f(\delta) = \sum_{(w,h) \in \mathbf{R}} | d_{w,h}^{(est)}(\boldsymbol{\delta}) - d_{w,h}^{(tgt)} |$ and iteratively searches for the optimal perturbation $\boldsymbol{\delta}^*$ under real-world conditions, demonstrating disappearance of object regions in multiple scenarios and even on a state-of-the-art model. The findings highlight practical threats to deployed MDE systems and motivate robust defenses and further research into quieter perturbations with reduced perceptibility.

Abstract

Deep neural networks (DNNs) remain vulnerable to adversarial attacks that cause misclassification when specific perturbations are added to input images. This vulnerability also threatens the reliability of DNN-based monocular depth estimation (MDE) models, making robustness enhancement a critical need in practical applications. To validate the vulnerability of DNN-based MDE models, this study proposes a projection-based adversarial attack method that projects perturbation light onto a target object. The proposed method employs physics-in-the-loop (PITL) optimization -- evaluating candidate solutions in actual environments to account for device specifications and disturbances -- and utilizes a distributed covariance matrix adaptation evolution strategy. Experiments confirmed that the proposed method successfully created adversarial examples that lead to depth misestimations, resulting in parts of objects disappearing from the target scene.

Figures (4)

• Figure 1: Overview of the proposed method.
• Figure 2: The proposed attack algorithm
• Figure 3: Exp. 1: Comparison between MOEA/D used in previous work RenyaDAIMO20232022MUL0001 and sep-CMA-ES used in our method.
• Figure 5: Exp. 3: results against other objects.