Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Physical Evaluation of Naturalistic Adversarial Patches for Camera-Based Traffic-Sign Detection

Brianna D'Urso, Tahmid Hasan Sakib, Syed Rafay Hasan, Terry N. Guo

TL;DR

Study of how well Naturalistic Adversarial Patches transfer to a physical traffic sign setting when the detector is trained on a customized dataset for an autonomous vehicle (AV) environment indicates the utility of CompGTSRB dataset and the proposed systematic physical protocols for credible patch evaluation.

Abstract

This paper studies how well Naturalistic Adversarial Patches (NAPs) transfer to a physical traffic sign setting when the detector is trained on a customized dataset for an autonomous vehicle (AV) environment. We construct a composite dataset, CompGTSRB (which is customized dataset for AV environment), by pasting traffic sign instances from the German Traffic Sign Recognition Benchmark (GTSRB) onto undistorted backgrounds captured from the target platform. CompGTSRB is used to train a YOLOv5 model and generate patches using a Generative Adversarial Network (GAN) with latent space optimization, following existing NAP methods. We carried out a series of experiments on our Quanser QCar testbed utilizing the front CSI camera provided in QCar. Across configurations, NAPs reduce the detector's STOP class confidence. Different configurations include distance, patch sizes, and patch placement. These results along with a detailed step-by-step methodology indicate the utility of CompGTSRB dataset and the proposed systematic physical protocols for credible patch evaluation. The research further motivate researching the defenses that address localized patch corruption in embedded perception pipelines.

Physical Evaluation of Naturalistic Adversarial Patches for Camera-Based Traffic-Sign Detection

TL;DR

Study of how well Naturalistic Adversarial Patches transfer to a physical traffic sign setting when the detector is trained on a customized dataset for an autonomous vehicle (AV) environment indicates the utility of CompGTSRB dataset and the proposed systematic physical protocols for credible patch evaluation.

Abstract

This paper studies how well Naturalistic Adversarial Patches (NAPs) transfer to a physical traffic sign setting when the detector is trained on a customized dataset for an autonomous vehicle (AV) environment. We construct a composite dataset, CompGTSRB (which is customized dataset for AV environment), by pasting traffic sign instances from the German Traffic Sign Recognition Benchmark (GTSRB) onto undistorted backgrounds captured from the target platform. CompGTSRB is used to train a YOLOv5 model and generate patches using a Generative Adversarial Network (GAN) with latent space optimization, following existing NAP methods. We carried out a series of experiments on our Quanser QCar testbed utilizing the front CSI camera provided in QCar. Across configurations, NAPs reduce the detector's STOP class confidence. Different configurations include distance, patch sizes, and patch placement. These results along with a detailed step-by-step methodology indicate the utility of CompGTSRB dataset and the proposed systematic physical protocols for credible patch evaluation. The research further motivate researching the defenses that address localized patch corruption in embedded perception pipelines.
Paper Structure (9 sections, 5 figures, 1 table)

This paper contains 9 sections, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. Workflow Overview
  4. Platform and Camera Calibration
  5. Composite Dataset Construction (CompGTSRB)
  6. Detector Training for Attack and Deployment
  7. Naturalistic Adversarial Patch Generation
  8. Physical Evaluation Protocol
  9. Experimental Results and Discussion

Figures (5)

  • Figure 1: Background capture and composite dataset (CompGTSRB) generation, followed by YOLOv5 training for attack and deployment models using the same composite data.
  • Figure 2: GTSRB dataset statistics. (a) Class instance counts indicate imbalance. (b) maRGB distribution shows illumination skew that motivates brightness matching in CompGTSRB.
  • Figure 3: NAP generation following Hu et al. b4. A GAN latent vector is optimized to produce a printable patch that lowers detector confidence.
  • Figure 4: Physical setup on the Quanser QCar. Front CSI camera observes a printed stop sign with a mounted patch. Distance d is varied; YOLOv5n runs onboard..
  • Figure 5: Mean YOLOv5n STOP confidence versus distance for clean and patched signs. Results include patch type, size, and placement combinations. Each point is a 15 s mean.