Universal Multi-view Black-box Attack against Object Detectors via Layout Optimization

TL;DR

The paper addresses the vulnerability of object detectors to adversarial examples in multi-view scenarios by introducing a universal multi-view black-box attack that optimizes a universal UV texture composed of image stickers. It models sticker placement as a circle-based layout with overlap and visibility constraints and solves it via a random-search method enhanced by an important-aware selection strategy, using a photo-realistic Unreal Engine evaluation tool for fair assessment. The approach yields substantial degradation in detection performance across four detectors (average P@0.5 drop of 74.29%) and demonstrates extensibility to warplane and traffic sign recognition, highlighting practical risks for physical-world deployments. This work provides a practical attack framework with strong multi-view effectiveness, detailed ablations, and a reusable evaluation tool for texture-based adversarial attacks.

Abstract

Object detectors have demonstrated vulnerability to adversarial examples crafted by small perturbations that can deceive the object detector. Existing adversarial attacks mainly focus on white-box attacks and are merely valid at a specific viewpoint, while the universal multi-view black-box attack is less explored, limiting their generalization in practice. In this paper, we propose a novel universal multi-view black-box attack against object detectors, which optimizes a universal adversarial UV texture constructed by multiple image stickers for a 3D object via the designed layout optimization algorithm. Specifically, we treat the placement of image stickers on the UV texture as a circle-based layout optimization problem, whose objective is to find the optimal circle layout filled with image stickers so that it can deceive the object detector under the multi-view scenario. To ensure reasonable placement of image stickers, two constraints are elaborately devised. To optimize the layout, we adopt the random search algorithm enhanced by the devised important-aware selection strategy to find the most appropriate image sticker for each circle from the image sticker pools. Extensive experiments conducted on four common object detectors suggested that the detection performance decreases by a large magnitude of 74.29% on average in multi-view scenarios. Additionally, a novel evaluation tool based on the photo-realistic simulator is designed to assess the texture-based attack fairly.

Figures (13)

• Figure 1: Example of vehicle graffiti in the real world.
• Figure 2: Illustration of sticker-based (a) and render-based (b-e) methods. From (b)-(e): (a) texture modified with the character; (b) right side position; (c) front position; (d) left side position. As we can see, only modifying a specific area (e.g., "A") will fail to attack the rendered image (d).
• Figure 3: Overview of the proposed method. The layout of adversarial patches in UV texture is optimized by choosing from the base elements (e.g., colored circle or square) or image sticker pool through layout constraints. With a physical renderer $\mathcal{R}$, the adversarial texture $T_{adv}$ is wrapped over the mesh $M$ of the 3D object and rendered to adversarial images $x_{adv}$, which are fed into the object detector $F$ for calculating fitness.
• Figure 4: Layout constraints. From left to right: (a) Overlap control: avoid the overlap of the circle; (b) Mask constraint: confine the circle in the appearance region of the 3D object.
• Figure 5: Illustration of replacing the circle with other elements (e.g., rectangle and nature image) in layout optimization. From left to right: left: circle and the largest inscribed square; middle: pure circle layout; right: natural image layout.