Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fall Leaf Adversarial Attack on Traffic Sign Classification

Anthony Etim, Jakub Szefer

TL;DR

Traffic sign classification in autonomous systems is vulnerable to adversarial perturbations, and this work proposes a novel fall leaf attack that uses natural leaves as inconspicuous perturbations to induce misclassification. The approach overlays leaves on signs, optimizing location, size, color, and orientation via a grid-search framework and customized leaf masks, evaluated on the LISA-CNN model. Results show high-confidence misclassifications for several signs, with strong dependence on leaf type and placement and measurable changes in edge-detection metrics that correlate with attack success. The study highlights a new real-world threat vector and motivates edge-aware defenses and robust TSR methods to ensure reliable autonomous perception under natural environmental perturbations.

Abstract

Adversarial input image perturbation attacks have emerged as a significant threat to machine learning algorithms, particularly in image classification setting. These attacks involve subtle perturbations to input images that cause neural networks to misclassify the input images, even though the images remain easily recognizable to humans. One critical area where adversarial attacks have been demonstrated is in automotive systems where traffic sign classification and recognition is critical, and where misclassified images can cause autonomous systems to take wrong actions. This work presents a new class of adversarial attacks. Unlike existing work that has focused on adversarial perturbations that leverage human-made artifacts to cause the perturbations, such as adding stickers, paint, or shining flashlights at traffic signs, this work leverages nature-made artifacts: tree leaves. By leveraging nature-made artifacts, the new class of attacks has plausible deniability: a fall leaf stuck to a street sign could come from a near-by tree, rather than be placed there by an malicious human attacker. To evaluate the new class of the adversarial input image perturbation attacks, this work analyses how fall leaves can cause misclassification in street signs. The work evaluates various leaves from different species of trees, and considers various parameters such as size, color due to tree leaf type, and rotation. The work demonstrates high success rate for misclassification. The work also explores the correlation between successful attacks and how they affect the edge detection, which is critical in many image classification algorithms.

Fall Leaf Adversarial Attack on Traffic Sign Classification

TL;DR

Traffic sign classification in autonomous systems is vulnerable to adversarial perturbations, and this work proposes a novel fall leaf attack that uses natural leaves as inconspicuous perturbations to induce misclassification. The approach overlays leaves on signs, optimizing location, size, color, and orientation via a grid-search framework and customized leaf masks, evaluated on the LISA-CNN model. Results show high-confidence misclassifications for several signs, with strong dependence on leaf type and placement and measurable changes in edge-detection metrics that correlate with attack success. The study highlights a new real-world threat vector and motivates edge-aware defenses and robust TSR methods to ensure reliable autonomous perception under natural environmental perturbations.

Abstract

Adversarial input image perturbation attacks have emerged as a significant threat to machine learning algorithms, particularly in image classification setting. These attacks involve subtle perturbations to input images that cause neural networks to misclassify the input images, even though the images remain easily recognizable to humans. One critical area where adversarial attacks have been demonstrated is in automotive systems where traffic sign classification and recognition is critical, and where misclassified images can cause autonomous systems to take wrong actions. This work presents a new class of adversarial attacks. Unlike existing work that has focused on adversarial perturbations that leverage human-made artifacts to cause the perturbations, such as adding stickers, paint, or shining flashlights at traffic signs, this work leverages nature-made artifacts: tree leaves. By leveraging nature-made artifacts, the new class of attacks has plausible deniability: a fall leaf stuck to a street sign could come from a near-by tree, rather than be placed there by an malicious human attacker. To evaluate the new class of the adversarial input image perturbation attacks, this work analyses how fall leaves can cause misclassification in street signs. The work evaluates various leaves from different species of trees, and considers various parameters such as size, color due to tree leaf type, and rotation. The work demonstrates high success rate for misclassification. The work also explores the correlation between successful attacks and how they affect the edge detection, which is critical in many image classification algorithms.

Paper Structure

This paper contains 20 sections, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Background
  4. Adversarial Attacks
  5. LISA-CNN and Traffic Sign Dataset
  6. Threat Model
  7. Fall Leaf Adversarial Attacks
  8. Leaf Attack
  9. Optimal Position Search Using Grid Search
  10. Patch Ratios
  11. Rotation Angles
  12. Leaf Images
  13. Custom Leaf Masks
  14. Attack Results
  15. Edge Detection
  16. ...and 5 more sections

Figures (5)

  • Figure 1: Test images used in evaluation of the attack.
  • Figure 2: Leaf masks generated by our attack method, one for each test image.
  • Figure 3: Leaf images used in evaluation of the attacks.
  • Figure 4: Adversarial images used in evaluation of the attacks. For each street sign and leaf type, only the adversarial image with highest confidence score is shown, all other images are omitted due to limited space.
  • Figure 5: Edge detection of attack images.