Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Time Traveling to Defend Against Adversarial Example Attacks in Image Classification

Anthony Etim, Jakub Szefer

TL;DR

Adversarial examples pose a major risk to traffic sign classification in autonomous systems. The authors propose a time-travel defense that leverages historical Street View imagery to compare current signs with past appearances, using majority voting to detect manipulations in real time. The defense is demonstrated on a LISA-CNN traffic sign classifier, achieving 100% defense effectiveness against the latest Street View shadow attacks and outperforming adversarial training baselines. This data-driven approach enables robust real-world deployment where historical imagery is accessible and highlights the value of historical context in enhancing adversarial robustness for autonomous driving.

Abstract

Adversarial example attacks have emerged as a critical threat to machine learning. Adversarial attacks in image classification abuse various, minor modifications to the image that confuse the image classification neural network -- while the image still remains recognizable to humans. One important domain where the attacks have been applied is in the automotive setting with traffic sign classification. Researchers have demonstrated that adding stickers, shining light, or adding shadows are all different means to make machine learning inference algorithms mis-classify the traffic signs. This can cause potentially dangerous situations as a stop sign is recognized as a speed limit sign causing vehicles to ignore it and potentially leading to accidents. To address these attacks, this work focuses on enhancing defenses against such adversarial attacks. This work shifts the advantage to the user by introducing the idea of leveraging historical images and majority voting. While the attacker modifies a traffic sign that is currently being processed by the victim's machine learning inference, the victim can gain advantage by examining past images of the same traffic sign. This work introduces the notion of ''time traveling'' and uses historical Street View images accessible to anybody to perform inference on different, past versions of the same traffic sign. In the evaluation, the proposed defense has 100% effectiveness against latest adversarial example attack on traffic sign classification algorithm.

Time Traveling to Defend Against Adversarial Example Attacks in Image Classification

TL;DR

Adversarial examples pose a major risk to traffic sign classification in autonomous systems. The authors propose a time-travel defense that leverages historical Street View imagery to compare current signs with past appearances, using majority voting to detect manipulations in real time. The defense is demonstrated on a LISA-CNN traffic sign classifier, achieving 100% defense effectiveness against the latest Street View shadow attacks and outperforming adversarial training baselines. This data-driven approach enables robust real-world deployment where historical imagery is accessible and highlights the value of historical context in enhancing adversarial robustness for autonomous driving.

Abstract

Adversarial example attacks have emerged as a critical threat to machine learning. Adversarial attacks in image classification abuse various, minor modifications to the image that confuse the image classification neural network -- while the image still remains recognizable to humans. One important domain where the attacks have been applied is in the automotive setting with traffic sign classification. Researchers have demonstrated that adding stickers, shining light, or adding shadows are all different means to make machine learning inference algorithms mis-classify the traffic signs. This can cause potentially dangerous situations as a stop sign is recognized as a speed limit sign causing vehicles to ignore it and potentially leading to accidents. To address these attacks, this work focuses on enhancing defenses against such adversarial attacks. This work shifts the advantage to the user by introducing the idea of leveraging historical images and majority voting. While the attacker modifies a traffic sign that is currently being processed by the victim's machine learning inference, the victim can gain advantage by examining past images of the same traffic sign. This work introduces the notion of ''time traveling'' and uses historical Street View images accessible to anybody to perform inference on different, past versions of the same traffic sign. In the evaluation, the proposed defense has 100% effectiveness against latest adversarial example attack on traffic sign classification algorithm.

Paper Structure

This paper contains 17 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Background
  4. Adversarial Attacks
  5. LISA-CNN and Traffic Sign Dataset
  6. Threat Model
  7. Adversarial Attack on Street Signs
  8. Shadow Attack
  9. Custom Shadow Masks
  10. Attack on Street View Images
  11. Defense: Time Traveling to Defend Against Adversarial Example Attacks
  12. Accessing Past Traffic Sign Images
  13. Classification of Past Traffic Sign Images
  14. Defense Results
  15. Discussion and Limitations
  16. ...and 2 more sections

Figures (4)

  • Figure 1: Test images used in evaluation of the attacks and defenses.
  • Figure 2: Shadow masks generated by our enhanced attack method, one for each test image.
  • Figure 3: Attack images which include the adversarial shadows added within the previously computed mask regions.
  • Figure 4: Historical images of the 5 test street signs used in evaluating the defense. Historical data is not always available for the same dates for each image, thus 3 prior images of each sign were used, resulting in some images being from different dates.