AnywhereDoor: Multi-Target Backdoor Attacks on Object Detection

TL;DR

AnywhereDoor tackles backdoor vulnerabilities in object detection by enabling multi-target, inference-time control. It achieves this through three innovations—objective disentanglement, trigger mosaicking, and strategic batching—allowing a single backdoored detector to perform removal, generation, and mislabeling across many targets while preserving clean mAP. Empirical results across Faster R-CNN, DETR, and YOLOv3 on VOC and COCO show high attack success rates and superior performance to baselines, with some resilience to defenses but clear trade-offs when defenses are applied. The work highlights significant security implications for safety-critical applications and motivates the development of robust defenses for object detectors.

Abstract

As object detection becomes integral to many safety-critical applications, understanding its vulnerabilities is essential. Backdoor attacks, in particular, pose a serious threat by implanting hidden triggers in victim models, which adversaries can later exploit to induce malicious behaviors during inference. However, current understanding is limited to single-target attacks, where adversaries must define a fixed malicious behavior (target) before training, making inference-time adaptability impossible. Given the large output space of object detection (including object existence prediction, bounding box estimation, and classification), the feasibility of flexible, inference-time model control remains unexplored. This paper introduces AnywhereDoor, a multi-target backdoor attack for object detection. Once implanted, AnywhereDoor allows adversaries to make objects disappear, fabricate new ones, or mislabel them, either across all object classes or specific ones, offering an unprecedented degree of control. This flexibility is enabled by three key innovations: (i) objective disentanglement to scale the number of supported targets; (ii) trigger mosaicking to ensure robustness even against region-based detectors; and (iii) strategic batching to address object-level data imbalances that hinder manipulation. Extensive experiments demonstrate that AnywhereDoor grants attackers a high degree of control, improving attack success rates by 26% compared to adaptations of existing methods for such flexible control.

Figures (10)

• Figure 1: Once implanted with AnywhereDoor, attackers can control the victim model with an unprecedented degree of freedom.
• Figure 2: Existing backdoor methods like Marksman doan2022marksman and BadNet gu2017badnets are either not effective enough in object detection or are not scalable to support a large number of attack targets. In contrast, AnywhereDoor remains effective even when granting the attacker a high degree of control.
• Figure 3: AnywhereDoor consists of three key modules: (a) objective disentanglement, which generates the trigger based on the attacker's desired target; (b) trigger mosaicking, which overlays the trigger on a clean input by tiling to ensure its effectiveness in any subregions; and (c) strategic batching, which forms batches dynamically during training. Once trained, the victim object detector exhibits desired malicious behaviors when the trigger presents while outputting normal detection results when receiving a clean image as input.
• Figure 4: Five attack scenarios (b-f) supported by AnywhereDoor with their poisoned labels on a training sample as an example.
• Figure 5: Objective disentanglement decomposes attack targets into two components: (i) removal and (ii) generation. The attacker configures $\boldsymbol{e}_r$ and $\boldsymbol{e}_g$ according to the desired target. Then, the two sub-models of the trigger generator will produce the corresponding triggers and combine them to generate the final trigger.