Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Device-aware Optical Adversarial Attack for a Portable Projector-camera System

Ning Jiang, Yanhong Liu, Dingheng Zeng, Yue Feng, Weihong Deng, Ying Li

TL;DR

This work tackles the vulnerability of face recognition systems to physical adversarial attacks delivered via a portable projector-camera setup. It introduces device-aware adaptations—namely, a patch-based resolution-aware loss and color-aware strategies with grayscale processing and dynamic color-mapping intervals—to bridge the gap between digital adversarial patterns and their physical projections. The method uses a universal adversarial mask applied across multiple views and optimizes a loss that combines cosine similarity, smoothness, and patch-consistency terms under perturbation constraints. Experimental results across spoof and real adversaries, multiple FR models, and commercial systems show strong physical attack performance, with an average digital-to-physical score degradation of about 14% and high attack success, highlighting practical security implications for FR deployments.

Abstract

Deep-learning-based face recognition (FR) systems are susceptible to adversarial examples in both digital and physical domains. Physical attacks present a greater threat to deployed systems as adversaries can easily access the input channel, allowing them to provide malicious inputs to impersonate a victim. This paper addresses the limitations of existing projector-camera-based adversarial light attacks in practical FR setups. By incorporating device-aware adaptations into the digital attack algorithm, such as resolution-aware and color-aware adjustments, we mitigate the degradation from digital to physical domains. Experimental validation showcases the efficacy of our proposed algorithm against real and spoof adversaries, achieving high physical similarity scores in FR models and state-of-the-art commercial systems. On average, there is only a 14% reduction in scores from digital to physical attacks, with high attack success rate in both white- and black-box scenarios.

Device-aware Optical Adversarial Attack for a Portable Projector-camera System

TL;DR

This work tackles the vulnerability of face recognition systems to physical adversarial attacks delivered via a portable projector-camera setup. It introduces device-aware adaptations—namely, a patch-based resolution-aware loss and color-aware strategies with grayscale processing and dynamic color-mapping intervals—to bridge the gap between digital adversarial patterns and their physical projections. The method uses a universal adversarial mask applied across multiple views and optimizes a loss that combines cosine similarity, smoothness, and patch-consistency terms under perturbation constraints. Experimental results across spoof and real adversaries, multiple FR models, and commercial systems show strong physical attack performance, with an average digital-to-physical score degradation of about 14% and high attack success, highlighting practical security implications for FR deployments.

Abstract

Deep-learning-based face recognition (FR) systems are susceptible to adversarial examples in both digital and physical domains. Physical attacks present a greater threat to deployed systems as adversaries can easily access the input channel, allowing them to provide malicious inputs to impersonate a victim. This paper addresses the limitations of existing projector-camera-based adversarial light attacks in practical FR setups. By incorporating device-aware adaptations into the digital attack algorithm, such as resolution-aware and color-aware adjustments, we mitigate the degradation from digital to physical domains. Experimental validation showcases the efficacy of our proposed algorithm against real and spoof adversaries, achieving high physical similarity scores in FR models and state-of-the-art commercial systems. On average, there is only a 14% reduction in scores from digital to physical attacks, with high attack success rate in both white- and black-box scenarios.
Paper Structure (11 sections, 3 equations, 3 figures, 3 tables, 1 algorithm)

This paper contains 11 sections, 3 equations, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. THE PROPOSED METHOD
  3. Problem Setup
  4. Device-aware Adversarial Pattern Generation
  5. Experiments
  6. Experimental Setup
  7. Results on Spoof Adversaries
  8. Results on Real Adversaries
  9. Ablation Experiments
  10. Commercial Systems
  11. CONCLUSIONS

Figures (3)

  • Figure 1: The overall framework.
  • Figure 2: Rough estimation of projector radiometric response for channel B.
  • Figure 3: Example of an impersonation attack on ArcFace in a white-box setting. Images (a) and (b) show a plastic head mode under projected adversarial light, both recognized as the target (c).