Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FIGhost: Fluorescent Ink-based Stealthy and Flexible Backdoor Attacks on Physical Traffic Sign Recognition

Shuai Yuan, Guowen Xu, Hongwei Li, Rui Zhang, Xinyuan Qian, Wenbo Jiang, Hangcheng Cao, Qingchuan Zhao

TL;DR

This work introduces FIGhost, a stealthy physical-world backdoor for traffic sign recognition that uses fluorescent ink triggered by UV light to remotely activate misbehavior in both traditional detectors and vision-language models. The approach combines graffiti-inspired trigger design, environment-aware augmentation, and automated sample generation to embed backdoors while preserving model performance on benign inputs. Empirical results show high attack success across models under real-world conditions, with robustness to environmental factors and resistance to common defenses. The findings highlight serious security risks in autonomous driving perception and stress the need for defense strategies specifically addressing fluorescence-based, UV-activated backdoors.

Abstract

Traffic sign recognition (TSR) systems are crucial for autonomous driving but are vulnerable to backdoor attacks. Existing physical backdoor attacks either lack stealth, provide inflexible attack control, or ignore emerging Vision-Large-Language-Models (VLMs). In this paper, we introduce FIGhost, the first physical-world backdoor attack leveraging fluorescent ink as triggers. Fluorescent triggers are invisible under normal conditions and activated stealthily by ultraviolet light, providing superior stealthiness, flexibility, and untraceability. Inspired by real-world graffiti, we derive realistic trigger shapes and enhance their robustness via an interpolation-based fluorescence simulation algorithm. Furthermore, we develop an automated backdoor sample generation method to support three attack objectives. Extensive evaluations in the physical world demonstrate FIGhost's effectiveness against state-of-the-art detectors and VLMs, maintaining robustness under environmental variations and effectively evading existing defenses.

FIGhost: Fluorescent Ink-based Stealthy and Flexible Backdoor Attacks on Physical Traffic Sign Recognition

TL;DR

This work introduces FIGhost, a stealthy physical-world backdoor for traffic sign recognition that uses fluorescent ink triggered by UV light to remotely activate misbehavior in both traditional detectors and vision-language models. The approach combines graffiti-inspired trigger design, environment-aware augmentation, and automated sample generation to embed backdoors while preserving model performance on benign inputs. Empirical results show high attack success across models under real-world conditions, with robustness to environmental factors and resistance to common defenses. The findings highlight serious security risks in autonomous driving perception and stress the need for defense strategies specifically addressing fluorescence-based, UV-activated backdoors.

Abstract

Traffic sign recognition (TSR) systems are crucial for autonomous driving but are vulnerable to backdoor attacks. Existing physical backdoor attacks either lack stealth, provide inflexible attack control, or ignore emerging Vision-Large-Language-Models (VLMs). In this paper, we introduce FIGhost, the first physical-world backdoor attack leveraging fluorescent ink as triggers. Fluorescent triggers are invisible under normal conditions and activated stealthily by ultraviolet light, providing superior stealthiness, flexibility, and untraceability. Inspired by real-world graffiti, we derive realistic trigger shapes and enhance their robustness via an interpolation-based fluorescence simulation algorithm. Furthermore, we develop an automated backdoor sample generation method to support three attack objectives. Extensive evaluations in the physical world demonstrate FIGhost's effectiveness against state-of-the-art detectors and VLMs, maintaining robustness under environmental variations and effectively evading existing defenses.
Paper Structure (21 sections, 5 equations, 13 figures, 7 tables)

This paper contains 21 sections, 5 equations, 13 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Traffic sign recognition
  4. Backdoor attack
  5. Threat model
  6. Method
  7. Overview
  8. Graffiti analysis
  9. Environment-aware trigger augmentation
  10. Automated sample generation
  11. Backdoor embedding
  12. Physical Evaluation
  13. Experimental setup
  14. Performance
  15. Ablation studies
  16. ...and 6 more sections

Figures (13)

  • Figure 1: The workflow of our FIGhost.
  • Figure 2: Triggers at different distances and UV intensities under various ambient light.
  • Figure 4: Physical backdoor attack examples on TSR for three attack goals.
  • Figure 5: The ASR of FIGhost on MiniGPT-4 when poison responses are generated using LLaVA-1.5 and GPT-4o.
  • Figure : (a) The maximum heart that remains within various traffic signs.
  • ...and 8 more figures