Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generative Adversarial Patches for Physical Attacks on Cross-Modal Pedestrian Re-Identification

Yue Su, Hao Li, Maoguo Gong

TL;DR

This paper introduces the first physical adversarial attack against VI-ReID models, and tests the models' ability to leverage deep-level implicit features by focusing on edge information, the most salient explicit feature differentiating individuals across modalities.

Abstract

Visible-infrared pedestrian Re-identification (VI-ReID) aims to match pedestrian images captured by infrared cameras and visible cameras. However, VI-ReID, like other traditional cross-modal image matching tasks, poses significant challenges due to its human-centered nature. This is evidenced by the shortcomings of existing methods, which struggle to extract common features across modalities, while losing valuable information when bridging the gap between them in the implicit feature space, potentially compromising security. To address this vulnerability, this paper introduces the first physical adversarial attack against VI-ReID models. Our method, termed Edge-Attack, specifically tests the models' ability to leverage deep-level implicit features by focusing on edge information, the most salient explicit feature differentiating individuals across modalities. Edge-Attack utilizes a novel two-step approach. First, a multi-level edge feature extractor is trained in a self-supervised manner to capture discriminative edge representations for each individual. Second, a generative model based on Vision Transformer Generative Adversarial Networks (ViTGAN) is employed to generate adversarial patches conditioned on the extracted edge features. By applying these patches to pedestrian clothing, we create realistic, physically-realizable adversarial samples. This black-box, self-supervised approach ensures the generalizability of our attack against various VI-ReID models. Extensive experiments on SYSU-MM01 and RegDB datasets, including real-world deployments, demonstrate the effectiveness of Edge- Attack in significantly degrading the performance of state-of-the-art VI-ReID methods.

Generative Adversarial Patches for Physical Attacks on Cross-Modal Pedestrian Re-Identification

TL;DR

This paper introduces the first physical adversarial attack against VI-ReID models, and tests the models' ability to leverage deep-level implicit features by focusing on edge information, the most salient explicit feature differentiating individuals across modalities.

Abstract

Visible-infrared pedestrian Re-identification (VI-ReID) aims to match pedestrian images captured by infrared cameras and visible cameras. However, VI-ReID, like other traditional cross-modal image matching tasks, poses significant challenges due to its human-centered nature. This is evidenced by the shortcomings of existing methods, which struggle to extract common features across modalities, while losing valuable information when bridging the gap between them in the implicit feature space, potentially compromising security. To address this vulnerability, this paper introduces the first physical adversarial attack against VI-ReID models. Our method, termed Edge-Attack, specifically tests the models' ability to leverage deep-level implicit features by focusing on edge information, the most salient explicit feature differentiating individuals across modalities. Edge-Attack utilizes a novel two-step approach. First, a multi-level edge feature extractor is trained in a self-supervised manner to capture discriminative edge representations for each individual. Second, a generative model based on Vision Transformer Generative Adversarial Networks (ViTGAN) is employed to generate adversarial patches conditioned on the extracted edge features. By applying these patches to pedestrian clothing, we create realistic, physically-realizable adversarial samples. This black-box, self-supervised approach ensures the generalizability of our attack against various VI-ReID models. Extensive experiments on SYSU-MM01 and RegDB datasets, including real-world deployments, demonstrate the effectiveness of Edge- Attack in significantly degrading the performance of state-of-the-art VI-ReID methods.

Paper Structure

This paper contains 14 sections, 6 equations, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. VI-ReID
  4. Adversarial Attack on Re-ID
  5. Methodology
  6. Problem Definition
  7. Edge Feature Extraction
  8. Adversarial Patch Generation
  9. Attack
  10. Experiments
  11. Attack Performance
  12. Ablation Study
  13. Real-world Experiment
  14. Conclusion

Figures (3)

  • Figure 1: VI-ReID has two query methods: one is to match the queried pedestrian in the infrared scene from the visible scene, and the other is to match the queried pedestrian in the visible scene from the infrared scene. Generally speaking, the model will give the image sequence with the highest matching degree. Our method aims to make sure that there are no correctly matched targets in the model output sequence.
  • Figure 2: The figure shows an overview of the Edge-Attack method. The edge feature extractor obtains fine-grained edge features that can be used to distinguish pedestrian identities by fusing multi-level edge features. Our patch generator uses this feature to generate adversarial patches, and uses the feature extractor as a discriminator to generate reverse edge features, thereby creating our target adversarial sample.
  • Figure 3: Our physical experiment scene setting. We design the adversarial IDs into clothes by using patches generated by Edge-Attack. IDs wearing these clothes will experience two modes of search on the AGW baseline: VIS TO IR (first two rows) and IR TO VIS (last row). The model will give a matching sequence, and our experiments have proven that it can make the ranking of the correctly matched IDs lag far behind, causing a powerful attack on the VI-ReID model.