CapGen:An Environment-Adaptive Generator of Adversarial Patches

TL;DR

The paper tackles the practical challenge of physical adversarial patches that must blend into real-world backgrounds while remaining effective against detection systems. It introduces CAPGen, which uses environment-derived base colors and a color probability matrix to generate camouflage patches that maintain adversarial strength. The study finds that pattern information dominates the attack performance over color, and proposes a fast adaptation method that replaces patch colors to match new backgrounds without re-optimizing from scratch. Extensive experiments on INRIA and FLIR_ADAS demonstrate strong stealthiness and competitive attack performance in both white-box and black-box settings, confirming the approach’s practical value for rapid, camouflage-aware physical attacks. Overall, the work pioneers a targeted exploration of patch components and environment alignment to enable rapid, concealment-focused adversarial patch generation.

Abstract

Adversarial patches, often used to provide physical stealth protection for critical assets and assess perception algorithm robustness, usually neglect the need for visual harmony with the background environment, making them easily noticeable. Moreover, existing methods primarily concentrate on improving attack performance, disregarding the intricate dynamics of adversarial patch elements. In this work, we introduce the Camouflaged Adversarial Pattern Generator (CAPGen), a novel approach that leverages specific base colors from the surrounding environment to produce patches that seamlessly blend with their background for superior visual stealthiness while maintaining robust adversarial performance. We delve into the influence of both patterns (i.e., color-agnostic texture information) and colors on the effectiveness of attacks facilitated by patches, discovering that patterns exert a more pronounced effect on performance than colors. Based on these findings, we propose a rapid generation strategy for adversarial patches. This involves updating the colors of high-performance adversarial patches to align with those of the new environment, ensuring visual stealthiness without compromising adversarial impact. This paper is the first to comprehensively examine the roles played by patterns and colors in the context of adversarial patches.

Figures (9)

• Figure 1: Detection results of different Adversarial coat against Yolov5s. CAPGen-based coat (ours) can better blend with the environment compared to AdvPatch physical2-based coat.
• Figure 2: Comparison between adversarial patch (Left), traditional camouflage (Mid) and our proposed method (Right). Adversarial patch can fool AI detector, but is not natural to human. Traditional camouflageyang2020research can fool human observer, but could be detected by AI detectors. Our proposed method can fool human observer and AI detector simultaneously.
• Figure 3: The pipeline of the proposed CAPGen. During the training stage, we optimize a color probability matrix to determine the probability of each pixel corresponding to each base color, which can generate adversarial patches that are consistent with the environment.
• Figure 4: Adversarial patches obtained by attacking Yolov5s with AdvPatch (\ref{['3.3a']}), its variants after changing base colors (\ref{['3.3b']}, \ref{['3.3c']}) and its variants after changing color probability matrix (\ref{['3.3d']}, \ref{['3.3e']}).
• Figure 5: The results of adversarial patches with increasing size (left) and the results with different numbers of base colors (right). All these adversarial patches are generated and tested on Yolov5s.