Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Steal Now and Attack Later: Evaluating Robustness of Object Detection against Black-box Adversarial Attacks

Erh-Chung Chen, Pin-Yu Chen, I-Hsin Chung, Che-Rung Lee

TL;DR

Steal Now and Attack Later addresses the vulnerability of object detectors to latency-based adversarial attacks in a black-box setting by forcing the model to predict numerous ghost objects, thereby increasing inference time within an $L_ extinf$ perturbation budget. The method builds a data-driven, patch-based attack pipeline—data collection from open-world datasets, position-centric object insertion, and color-space perturbation projected onto the $L_ extinf$ ball with the perturbation bounded by $\epsilon d$—to craft adversarial examples without model access. Empirical results show successful attacks across diverse detectors (e.g., Faster R-CNN, RetinaNet, FCOS, YOLOv8, DERT) and public vision APIs (GCP, Azure) with ASR rising with larger perturbations and modest data collection costs (roughly $3 on local GPU and under $1 per API query). The work highlights practical security implications and discusses defenses such as multi-dimension inference, context consistency checks, and image-quality screening, while advocating private locally deployed models as an economical mitigation.

Abstract

Latency attacks against object detection represent a variant of adversarial attacks that aim to inflate the inference time by generating additional ghost objects in a target image. However, generating ghost objects in the black-box scenario remains a challenge since information about these unqualified objects remains opaque. In this study, we demonstrate the feasibility of generating ghost objects in adversarial examples by extending the concept of "steal now, decrypt later" attacks. These adversarial examples, once produced, can be employed to exploit potential vulnerabilities in the AI service, giving rise to significant security concerns. The experimental results demonstrate that the proposed attack achieves successful attacks across various commonly used models and Google Vision API without any prior knowledge about the target model. Additionally, the average cost of each attack is less than \$ 1 dollars, posing a significant threat to AI security.

Steal Now and Attack Later: Evaluating Robustness of Object Detection against Black-box Adversarial Attacks

TL;DR

Steal Now and Attack Later addresses the vulnerability of object detectors to latency-based adversarial attacks in a black-box setting by forcing the model to predict numerous ghost objects, thereby increasing inference time within an perturbation budget. The method builds a data-driven, patch-based attack pipeline—data collection from open-world datasets, position-centric object insertion, and color-space perturbation projected onto the ball with the perturbation bounded by —to craft adversarial examples without model access. Empirical results show successful attacks across diverse detectors (e.g., Faster R-CNN, RetinaNet, FCOS, YOLOv8, DERT) and public vision APIs (GCP, Azure) with ASR rising with larger perturbations and modest data collection costs (roughly 1 per API query). The work highlights practical security implications and discusses defenses such as multi-dimension inference, context consistency checks, and image-quality screening, while advocating private locally deployed models as an economical mitigation.

Abstract

Latency attacks against object detection represent a variant of adversarial attacks that aim to inflate the inference time by generating additional ghost objects in a target image. However, generating ghost objects in the black-box scenario remains a challenge since information about these unqualified objects remains opaque. In this study, we demonstrate the feasibility of generating ghost objects in adversarial examples by extending the concept of "steal now, decrypt later" attacks. These adversarial examples, once produced, can be employed to exploit potential vulnerabilities in the AI service, giving rise to significant security concerns. The experimental results demonstrate that the proposed attack achieves successful attacks across various commonly used models and Google Vision API without any prior knowledge about the target model. Additionally, the average cost of each attack is less than \$ 1 dollars, posing a significant threat to AI security.
Paper Structure (21 sections, 3 equations, 10 figures, 3 tables, 2 algorithms)

This paper contains 21 sections, 3 equations, 10 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Object Detection
  4. Adversarial Attack
  5. Methods and Algorithms
  6. Threat Model and Motivations
  7. Attack Flow Overview
  8. Data Collection from Open World
  9. Position-centric Candidate Selection
  10. Color Manipulation
  11. Experiments
  12. Setup
  13. Performance Evaluation
  14. Ablation study
  15. Analysis of spatial distribution
  16. ...and 6 more sections

Figures (10)

  • Figure 1: The execution flow of object detection.
  • Figure 2: Attack Flow Overview.
  • Figure 3: Images output at each stage of the attack process. (\ref{['fig:atk_pipeline_0']}) is the original image; (\ref{['fig:atk_pipeline_1']}) shows the image with external objects; (\ref{['fig:atk_pipeline_2']}) displays the modified image projected onto the epsilon ball; and (\ref{['fig:atk_pipeline_3']}) showcases the final adversarial image. The predictions for both the original and the adversarial images are presented in (\ref{['fig:atk_pipeline_0_result']}) and (\ref{['fig:atk_pipeline_3_result']}), respectively.
  • Figure 4: The strength of adversarial examples highly depends on color fluctuations.
  • Figure 5: Objects tend to be clustered in specific regions.
  • ...and 5 more figures