Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Secure Traffic Sign Recognition: An Attention-Enabled Universal Image Inpainting Mechanism against Light Patch Attacks

Hangcheng Cao, Longzhi Yuan, Guowen Xu, Ziyang He, Zhengru Fang, Yuguang Fang

TL;DR

This work proposes a universal image inpainting mechanism, namely, SafeSign, that relies on attention-enabled multi-view image fusion to repair traffic signs contaminated by adversarial light patches, thereby ensuring the accurate sign recognition.

Abstract

Traffic sign recognition systems play a crucial role in assisting drivers to make informed decisions while driving. However, due to the heavy reliance on deep learning technologies, particularly for future connected and autonomous driving, these systems are susceptible to adversarial attacks that pose significant safety risks to both personal and public transportation. Notably, researchers recently identified a new attack vector to deceive sign recognition systems: projecting well-designed adversarial light patches onto traffic signs. In comparison with traditional adversarial stickers or graffiti, these emerging light patches exhibit heightened aggression due to their ease of implementation and outstanding stealthiness. To effectively counter this security threat, we propose a universal image inpainting mechanism, namely, SafeSign. It relies on attention-enabled multi-view image fusion to repair traffic signs contaminated by adversarial light patches, thereby ensuring the accurate sign recognition. Here, we initially explore the fundamental impact of malicious light patches on the local and global feature spaces of authentic traffic signs. Then, we design a binary mask-based U-Net image generation pipeline outputting diverse contaminated sign patterns, to provide our image inpainting model with needed training data. Following this, we develop an attention mechanism-enabled neural network to jointly utilize the complementary information from multi-view images to repair contaminated signs. Finally, extensive experiments are conducted to evaluate SafeSign's effectiveness in resisting potential light patch-based attacks, bringing an average accuracy improvement of 54.8% in three widely-used sign recognition models

Secure Traffic Sign Recognition: An Attention-Enabled Universal Image Inpainting Mechanism against Light Patch Attacks

TL;DR

This work proposes a universal image inpainting mechanism, namely, SafeSign, that relies on attention-enabled multi-view image fusion to repair traffic signs contaminated by adversarial light patches, thereby ensuring the accurate sign recognition.

Abstract

Traffic sign recognition systems play a crucial role in assisting drivers to make informed decisions while driving. However, due to the heavy reliance on deep learning technologies, particularly for future connected and autonomous driving, these systems are susceptible to adversarial attacks that pose significant safety risks to both personal and public transportation. Notably, researchers recently identified a new attack vector to deceive sign recognition systems: projecting well-designed adversarial light patches onto traffic signs. In comparison with traditional adversarial stickers or graffiti, these emerging light patches exhibit heightened aggression due to their ease of implementation and outstanding stealthiness. To effectively counter this security threat, we propose a universal image inpainting mechanism, namely, SafeSign. It relies on attention-enabled multi-view image fusion to repair traffic signs contaminated by adversarial light patches, thereby ensuring the accurate sign recognition. Here, we initially explore the fundamental impact of malicious light patches on the local and global feature spaces of authentic traffic signs. Then, we design a binary mask-based U-Net image generation pipeline outputting diverse contaminated sign patterns, to provide our image inpainting model with needed training data. Following this, we develop an attention mechanism-enabled neural network to jointly utilize the complementary information from multi-view images to repair contaminated signs. Finally, extensive experiments are conducted to evaluate SafeSign's effectiveness in resisting potential light patch-based attacks, bringing an average accuracy improvement of 54.8% in three widely-used sign recognition models
Paper Structure (19 sections, 9 equations, 18 figures)

This paper contains 19 sections, 9 equations, 18 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Adversarial Attack Against TSR
  4. Defense Mechanism for TSR
  5. Background and Preliminaries
  6. Traffic Sign Recognition
  7. Light Patch-based Attacks
  8. Why Is SafeSign Designed as a Preset Module?
  9. System Design
  10. System Overview
  11. Data Preprocessing
  12. Contaminated Sign Generation
  13. Sign Image Reconstruction
  14. Put All Things Together
  15. Evaluation
  16. ...and 4 more sections

Figures (18)

  • Figure 1: Illustration of SafeSign's role, that is, repairing signs contaminated by adversarial light patches. We take the "STOP" sign repair as an example: with enabling SafeSign, the contaminated sign pattern is repaired and then TSR can correctly recognize its label.
  • Figure 2: Illustration of traffic sign recognition workflows in (a) one-stage and (b) two-stage architectures.
  • Figure 3: Four common adversarial attack modes launched by (a) infrared spot sato2024invisible, (b) laser line duan2021adversarial, (c) artificial shadow zhong2022shadows, and (d) projection graffiti lovisotto2021slap.
  • Figure 4: The workflow of SafeSign, consisting of three parts that are data proprecessing, contaminated sign generation, and sign reconstruction.
  • Figure 5: Sign image augmentation using five common transformation operations on (a) a raw sign image, that are adjusting (b) rotation, (c) shearing, (d) brightness, (e) saturation, and (f) contrast.
  • ...and 13 more figures