Robust Physical Adversarial Patches Using Dynamically Optimized Clusters
Harrison Bagley, Will Meakin, Simon Lucey, Yee Wei Law, Tat-Jun Chin
TL;DR
This work targets the vulnerability of person detectors to physical adversarial patches under scale variation. It introduces a novel dynamic superpixel regularizer by backpropagating through SLIC via the Implicit Function Theorem, producing scale-resilient patch structures that maintain their adversarial signal under interpolation. The authors validate the approach with a reproducible physical evaluation protocol using screens and cardboard cut-outs, showing improved performance in both digital and physical domains and stronger transfer to unseen detectors than prior patches. The findings suggest that coarse, spatially structured attackers can achieve robust real-world efficacy, motivating further defenses and extensions to richer fabrications and transfer-based regularization.
Abstract
Physical adversarial attacks on deep learning systems is concerning due to the ease of deploying such attacks, usually by placing an adversarial patch in a scene to manipulate the outcomes of a deep learning model. Training such patches typically requires regularization that improves physical realizability (e.g., printability, smoothness) and/or robustness to real-world variability (e.g. deformations, viewing angle, noise). One type of variability that has received little attention is scale variability. When a patch is rescaled, either digitally through downsampling/upsampling or physically through changing imaging distances, interpolation-induced color mixing occurs. This smooths out pixel values, resulting in a loss of high-frequency patterns and degrading the adversarial signal. To address this, we present a novel superpixel-based regularization method that guides patch optimization to scale-resilient structures. Our ap proach employs the Simple Linear Iterative Clustering (SLIC) algorithm to dynamically cluster pixels in an adversarial patch during optimization. The Implicit Function Theorem is used to backpropagate gradients through SLIC to update the superpixel boundaries and color. This produces patches that maintain their structure over scale and are less susceptible to interpolation losses. Our method achieves greater performance in the digital domain, and when realized physically, these performance gains are preserved, leading to improved physical performance. Real-world performance was objectively assessed using a novel physical evaluation protocol that utilizes screens and cardboard cut-outs to systematically vary real-world conditions.