When Person Re-Identification Meets Event Camera: A Benchmark Dataset and An Attribute-guided Re-Identification Framework

TL;DR

This work tackles the scarcity and limitations of RGB-Event person re-identification by introducing EvReID, a large-scale dual-modality benchmark, and TriPro-ReID, an attribute-guided, cross-modal framework. TriPro-ReID integrates Cross-Modal Prompting (CMP) and Positive-Negative Attribute Prompts (PNAP) within a three-stage training regime to fuse RGB and Event features while leveraging pedestrian attributes via CLIP-based prompts. Empirical results on EvReID and MARS$^*$ demonstrate state-of-the-art performance and substantial gains from semantic prompts and multimodal alignment, validating the approach for robust, privacy-conscious ReID in diverse conditions. The work provides a solid data and methodological foundation for future RGB-Event ReID research and practical deployment, with release of code and dataset for community use.

Abstract

Recent researchers have proposed using event cameras for person re-identification (ReID) due to their promising performance and better balance in terms of privacy protection, event camera-based person ReID has attracted significant attention. Currently, mainstream event-based person ReID algorithms primarily focus on fusing visible light and event stream, as well as preserving privacy. Although significant progress has been made, these methods are typically trained and evaluated on small-scale or simulated event camera datasets, making it difficult to assess their real identification performance and generalization ability. To address the issue of data scarcity, this paper introduces a large-scale RGB-event based person ReID dataset, called EvReID. The dataset contains 118,988 image pairs and covers 1200 pedestrian identities, with data collected across multiple seasons, scenes, and lighting conditions. We also evaluate 15 state-of-the-art person ReID algorithms, laying a solid foundation for future research in terms of both data and benchmarking. Based on our newly constructed dataset, this paper further proposes a pedestrian attribute-guided contrastive learning framework to enhance feature learning for person re-identification, termed TriPro-ReID. This framework not only effectively explores the visual features from both RGB frames and event streams, but also fully utilizes pedestrian attributes as mid-level semantic features. Extensive experiments on the EvReID dataset and MARS datasets fully validated the effectiveness of our proposed RGB-Event person ReID framework. The benchmark dataset and source code will be released on https://github.com/Event-AHU/Neuromorphic_ReID

Figures (6)

• Figure 1: Imaging principle of event camera and comparison of RGB frames and event streams in challenging scenarios.
• Figure 2: Comparison of existing video-based ReID datasets and our newly proposed EvReID dataset.
• Figure 3: An illustration of our proposed TriProReId framework. The left part illustrates the overall architecture of the RGB-Event person re-identification framework, consisting of dual-stream backbones and multi-level fusion modules. The right part presents the three-stage training strategy, including Text Prompt Align, Multimodal Prompt Align, and Visual Modal Tuning stages.
• Figure 4: Visualization of pedestrian attribute prediction results. The first row shows the predictions from the MARS dataset, while the second row displays the predictions from the EvReID dataset.
• Figure 5: Visualization of pedestrian heatmap results. The first row shows the heatmap for the RGB modality, while the second row displays the corresponding heatmap for the Event modality.