Guided Diffusion-based Generation of Adversarial Objects for Real-World Monocular Depth Estimation Attacks

TL;DR

Monocular Depth Estimation (MDE) systems are vulnerable to physical adversarial attacks. The authors propose a training-free, diffusion-guided adversarial framework that generates scene-consistent objects via Salient Region Selection and Jacobian Vector Product Guidance (JVPG), guided by a weak text cue from a Vision-Language Model. The adversarial objective $L_{ ext{adv}} = \| f_{M_T}(z) - \lambda \cdot f_{M_T}(x) \|_2^2$ steers generation to distort depth predictions while preserving realism through a pre-trained diffusion prior and geometry-aware gradient modulation. Digital and real-world experiments demonstrate superior attack potency and stealth across multiple MDE architectures and validate physical deployability, underscoring safety implications and suggesting avenues for improving perception robustness.

Abstract

Monocular Depth Estimation (MDE) serves as a core perception module in autonomous driving systems, but it remains highly susceptible to adversarial attacks. Errors in depth estimation may propagate through downstream decision making and influence overall traffic safety. Existing physical attacks primarily rely on texture-based patches, which impose strict placement constraints and exhibit limited realism, thereby reducing their effectiveness in complex driving environments. To overcome these limitations, this work introduces a training-free generative adversarial attack framework that generates naturalistic, scene-consistent adversarial objects via a diffusion-based conditional generation process. The framework incorporates a Salient Region Selection module that identifies regions most influential to MDE and a Jacobian Vector Product Guidance mechanism that steers adversarial gradients toward update directions supported by the pre-trained diffusion model. This formulation enables the generation of physically plausible adversarial objects capable of inducing substantial adversarial depth shifts. Extensive digital and physical experiments demonstrate that our method significantly outperforms existing attacks in effectiveness, stealthiness, and physical deployability, underscoring its strong practical implications for autonomous driving safety assessment.

Figures (8)

• Figure 1: Comparison between our generative adversarial object attack and previous patch-based physical attacks. Previous patch-based methods are constrained to fixed spatial locations and rely on unnatural textures, which are prone to being detected and filtered by anomaly or out-of-distribution detectors. In contrast, our method leverages open-vocabulary object generation to generate natural object-level adversarial content that can be flexibly placed at any region.
• Figure 2: Overview of the proposed generative adversarial attack framework. The pipeline first performs Salient Region Selection by injecting perturbations into image regions, ranking their influence on the MDE model, and selecting the top-$K$ most vulnerable regions. In the second stage, a diffusion-based generator produces a scene-consistent adversarial object at the selected region, where Jacobian Vector Product Guidance (JVPG) injects adversarial gradients into the diffusion trajectory while preserving text-conditional semantics and visual realism, ultimately inducing substantial depth shifts in the MDE output.
• Figure 3: Visualization of salient region estimation across diverse driving scenes. The yellow bounding box denotes the target object mask $M_T$. Warmer colors indicate regions with higher saliency scores, which exert stronger influence on the depth predicted within $M_T$ and are therefore prioritized for adversarial object insertion, whereas cooler colors correspond to non-salient regions.
• Figure 4: Comparison of denoising trajectories under different Jacobian singular directions. The first row shows the original diffusion trajectory. The second row applies perturbations along the dominant singular direction $u^+$, which preserves coherent semantic structures. The third row applies perturbations along the smallest singular direction $u^-$, resulting in disordered, non-semantic artifacts. These visualizations highlight that $u^+$ corresponds to meaningful generative directions, whereas $u^-$ drives the diffusion process away from semantic consistency.
• Figure 5: Qualitative visualization of our generative adversarial object attack and its impact on MDE. From left to right: (a) original RGB scene, (b) adversarial scene with the generated object inserted, (c) predicted depth map for the original scene, (d) predicted depth map for the adversarial scene, and (e) depth difference map. The yellow bounding box marks the target region $M_T$, where the induced depth shift is evaluated. Brighter regions in the fifth column indicate larger depth deviations, highlighting that our method induces significant depth shifts while preserving realistic appearance in the digital domain.