MVPatch: More Vivid Patch for Adversarial Camouflaged Attacks on Object Detectors in the Physical World

TL;DR

A Dual-Perception-Based Framework (DPBF) is proposed to generate the More Vivid Patch (MVPatch), which enhances transferability, stealthiness, and practicality, and introduces naturalness and transferability scores to provide an unbiased assessment of APs.

Abstract

Recent studies have shown that Adversarial Patches (APs) can effectively manipulate object detection models. However, the conspicuous patterns often associated with these patches tend to attract human attention, posing a significant challenge. Existing research has primarily focused on enhancing attack efficacy in the physical domain while often neglecting the optimization of stealthiness and transferability. Furthermore, applying APs in real-world scenarios faces major challenges related to transferability, stealthiness, and practicality. To address these challenges, we introduce generalization theory into the context of APs, enabling our iterative process to simultaneously enhance transferability and refine visual correlation with realistic images. We propose a Dual-Perception-Based Framework (DPBF) to generate the More Vivid Patch (MVPatch), which enhances transferability, stealthiness, and practicality. The DPBF integrates two key components: the Model-Perception-Based Module (MPBM) and the Human-Perception-Based Module (HPBM), along with regularization terms. The MPBM employs ensemble strategy to reduce object confidence scores across multiple detectors, thereby improving AP transferability with robust theoretical support. Concurrently, the HPBM introduces a lightweight method for achieving visual similarity, creating natural and inconspicuous adversarial patches without relying on additional generative models. The regularization terms further enhance the practicality of the generated APs in the physical domain. Additionally, we introduce naturalness and transferability scores to provide an unbiased assessment of APs. Extensive experimental validation demonstrates that MVPatch achieves superior transferability and a natural appearance in both digital and physical domains, underscoring its effectiveness and stealthiness.

Figures (8)

• Figure 1: Introduction to the attack scenarios of MVPatch and illustration of the meaningful and meaningless adversarial patches. (a) demonstrates that MVPatch can make a person invisible to detectors in the real world, while (b) demonstrates that detectors can successfully distinguish a person in the digital world. (c) and (d) demonstrate both diverse meaningful adversarial patches b96 and meaningless adversarial patches b87b100, respectively.
• Figure 2: The MVPatch pipeline involves embedding masks into benign images and applying them to object detectors to determine the object confidence scores. To achieve improved attack performance, the algorithm minimizes various losses, such as TV loss, NPS loss, OBJ loss, and CSS loss, to obtain the optimal adversarial patch. Algorithm \ref{['a1']} illustrates the complete procedure for the MVPatch algorithm.
• Figure 3: The naturalness score (NS) of adversarial patches with diverse $\lambda$ parameters. As the NS increases, the level of similarity between the source image and the generated image rises.
• Figure 4: Adversarial patches are generated by various object detectors, including YOLOv2, YOLOv3, YOLOv3-tiny, YOLOv4, and YOLOv4-tiny and compared attack performance on the individual model with Natural Patchb96. Additionally, we compare the performance of our ensemble attack with that of the Natural Patch. The attack performance of diverse adversarial patches are illustrated as Table \ref{['t4']}.
• Figure 5: Adversarial patches, such as AdvPatchb87, AdvTextureb100, Natural Patchb96, and our MVPatch, are employed in the physical world using diverse attack methods. Table\ref{['t6']} displays the attack success rate of different adversarial patches, as well as other important evaluation metrics.

Theorems & Definitions (12)

• Definition 1: Adversarial Patch
• Definition 2: Generalization Error
• Definition 3: Ensemble Model
• Remark 1
• Theorem 1: Generalization Error of Ensemble Model
• proof
• Remark 2
• Theorem 2: Stability of the Ensemble Model
• proof
• Remark 3