Mask-based Invisible Backdoor Attacks on Object Detection

TL;DR

This work demonstrates a mask-based invisible backdoor attack on object detection, introducing three attack scenarios—Object Disappearance, Misclassification, and Generation (ODA, OMA, OGA)—and validating them across Faster R-CNN, YOLOv3, and YOLOv5. It formalizes a region-aware trigger mechanism via a transformation $T(x)=x+\mu(o)\odot g(x)$ with $||g(x)||_{\infty}\le \epsilon$, optimized jointly with the detector parameters to preserve normal performance while enabling malicious behavior upon trigger. The study reports high attack success rates across models and scenarios, while also evaluating defenses such as STRIP and Grad-CAM, finding STRIP ineffective for these invisible perturbations and Grad-CAM offering potential detection cues. The results underscore the practical risks of invisible backdoors in critical detection systems and motivate further development of robust defenses; code is made available at the provided repository.

Abstract

Deep learning models have achieved unprecedented performance in the domain of object detection, resulting in breakthroughs in areas such as autonomous driving and security. However, deep learning models are vulnerable to backdoor attacks. These attacks prompt models to behave similarly to standard models without a trigger; however, they act maliciously upon detecting a predefined trigger. Despite extensive research on backdoor attacks in image classification, their application to object detection remains relatively underexplored. Given the widespread application of object detection in critical real-world scenarios, the sensitivity and potential impact of these vulnerabilities cannot be overstated. In this study, we propose an effective invisible backdoor attack on object detection utilizing a mask-based approach. Three distinct attack scenarios were explored for object detection: object disappearance, object misclassification, and object generation attack. Through extensive experiments, we comprehensively examined the effectiveness of these attacks and tested certain defense methods to determine effective countermeasures. Code will be available at https://github.com/jeongjin0/invisible-backdoor-object-detection

Figures (3)

• Figure 1: Scenarios in our invisible backdoor attack. The columns categorize the different attack scenarios: the first column for ODA, the second for OMA, and the third for OGA. The first row demonstrates model predictions without backdoor triggers, whereas the second row lists predictions with the insertion of our invisible triggers specific to each scenario.
• Figure 2: Performance against STRIP
• Figure 3: Performance under GradCam heatmap