Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating Adversarial Attacks on Traffic Sign Classifiers beyond Standard Baselines

Svetlana Pavlitska, Leopold Müller, J. Marius Zöllner

TL;DR

This work tackles the problem of unfairly favoring fixed TSR baselines by decoupling architectures from datasets and evaluating attacks across both established baselines and generic CNNs. It introduces a two-phase, mask-based attack framework with a color-regularized loss and EOT-inspired robustness, comparing inconspicuous and visible perturbations in digital and real-world settings. Key findings show that traditional baselines (e.g., LISA-CNN, GTSRB-CNN) are more vulnerable than generic models, and that real-world performance can diverge from digital-domain results, highlighting the need for broader, cross-architecture evaluation in adversarial traffic sign research. The paper provides open-source code to facilitate reproducibility and encourages adopting diverse baselines in future threat-model assessments to better gauge practical risk.

Abstract

Adversarial attacks on traffic sign classification models were among the first successfully tried in the real world. Since then, the research in this area has been mainly restricted to repeating baseline models, such as LISA-CNN or GTSRB-CNN, and similar experiment settings, including white and black patches on traffic signs. In this work, we decouple model architectures from the datasets and evaluate on further generic models to make a fair comparison. Furthermore, we compare two attack settings, inconspicuous and visible, which are usually regarded without direct comparison. Our results show that standard baselines like LISA-CNN or GTSRB-CNN are significantly more susceptible than the generic ones. We, therefore, suggest evaluating new attacks on a broader spectrum of baselines in the future. Our code is available at \url{https://github.com/KASTEL-MobilityLab/attacks-on-traffic-sign-recognition/}.

Evaluating Adversarial Attacks on Traffic Sign Classifiers beyond Standard Baselines

TL;DR

This work tackles the problem of unfairly favoring fixed TSR baselines by decoupling architectures from datasets and evaluating attacks across both established baselines and generic CNNs. It introduces a two-phase, mask-based attack framework with a color-regularized loss and EOT-inspired robustness, comparing inconspicuous and visible perturbations in digital and real-world settings. Key findings show that traditional baselines (e.g., LISA-CNN, GTSRB-CNN) are more vulnerable than generic models, and that real-world performance can diverge from digital-domain results, highlighting the need for broader, cross-architecture evaluation in adversarial traffic sign research. The paper provides open-source code to facilitate reproducibility and encourages adopting diverse baselines in future threat-model assessments to better gauge practical risk.

Abstract

Adversarial attacks on traffic sign classification models were among the first successfully tried in the real world. Since then, the research in this area has been mainly restricted to repeating baseline models, such as LISA-CNN or GTSRB-CNN, and similar experiment settings, including white and black patches on traffic signs. In this work, we decouple model architectures from the datasets and evaluate on further generic models to make a fair comparison. Furthermore, we compare two attack settings, inconspicuous and visible, which are usually regarded without direct comparison. Our results show that standard baselines like LISA-CNN or GTSRB-CNN are significantly more susceptible than the generic ones. We, therefore, suggest evaluating new attacks on a broader spectrum of baselines in the future. Our code is available at \url{https://github.com/KASTEL-MobilityLab/attacks-on-traffic-sign-recognition/}.

Paper Structure

This paper contains 14 sections, 3 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Attack Approach
  4. Phase I - Mask Creation
  5. Phase II - Attack Generation
  6. Loss Function
  7. Inconspicuous vs. Visible Attack
  8. Evaluation
  9. Experiments and Evaluation
  10. Experimental Setup
  11. Performance on Clean Data
  12. Evaluation in the Digital Domain
  13. Real-World Evaluation
  14. Conclusion

Figures (4)

  • Figure 1: Inconspicuous (a) and visible (b) adversarial perturbations of a stop sign.
  • Figure 2: Stopzhong2022shadows and Speed limit 30km/h signs and the corresponding masks.
  • Figure 3: Baseline architectures. In CNN$_{small}$ and CNN$_{large}$, all Conv layers use the ReLU activation function. In CNN-STN, all Conv layers use the LeakyReLU activation function and batch normalization. The number of classes $n\_classes$ is 43 for GTRSDB and 16 for LISA.
  • Figure 4: Examples of inconspicuous and visible attacks on a Stop sign.