Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Base-Detail Feature Learning Framework for Visible-Infrared Person Re-Identification

Zhihao Gong, Lian Wu, Yong Xu

TL;DR

This work tackles VIReID by explicitly modeling both modality-shared base features and modality-specific detail features. It introduces the Base-Detail Feature Learning Framework (BDLF) with a Detail Feature Extraction (DFE) module and a Base Embedding Generation (BEG) block, guided by a Specific-Shared Knowledge Distillation (SKD) loss and correlation-based constraints to maximize complementary information across VIS and IR modalities. The approach, implemented on a ResNet-50 backbone with INN-based detail extraction and cross-modality fusion, achieves state-of-the-art results on SYSU-MM01, RegDB, and LLCM, demonstrating robustness to cross-modality gaps and challenging lighting conditions. Overall, BDLF provides a unified, end-to-end framework that leverages both shared and specific cues to improve VIReID performance with practical implications for 24-hour surveillance systems.

Abstract

Visible-infrared person re-identification (VIReID) provides a solution for ReID tasks in 24-hour scenarios; however, significant challenges persist in achieving satisfactory performance due to the substantial discrepancies between visible (VIS) and infrared (IR) modalities. Existing methods inadequately leverage information from different modalities, primarily focusing on digging distinguishing features from modality-shared information while neglecting modality-specific details. To fully utilize differentiated minutiae, we propose a Base-Detail Feature Learning Framework (BDLF) that enhances the learning of both base and detail knowledge, thereby capitalizing on both modality-shared and modality-specific information. Specifically, the proposed BDLF mines detail and base features through a lossless detail feature extraction module and a complementary base embedding generation mechanism, respectively, supported by a novel correlation restriction method that ensures the features gained by BDLF enrich both detail and base knowledge across VIS and IR features. Comprehensive experiments conducted on the SYSU-MM01, RegDB, and LLCM datasets validate the effectiveness of BDLF.

Base-Detail Feature Learning Framework for Visible-Infrared Person Re-Identification

TL;DR

This work tackles VIReID by explicitly modeling both modality-shared base features and modality-specific detail features. It introduces the Base-Detail Feature Learning Framework (BDLF) with a Detail Feature Extraction (DFE) module and a Base Embedding Generation (BEG) block, guided by a Specific-Shared Knowledge Distillation (SKD) loss and correlation-based constraints to maximize complementary information across VIS and IR modalities. The approach, implemented on a ResNet-50 backbone with INN-based detail extraction and cross-modality fusion, achieves state-of-the-art results on SYSU-MM01, RegDB, and LLCM, demonstrating robustness to cross-modality gaps and challenging lighting conditions. Overall, BDLF provides a unified, end-to-end framework that leverages both shared and specific cues to improve VIReID performance with practical implications for 24-hour surveillance systems.

Abstract

Visible-infrared person re-identification (VIReID) provides a solution for ReID tasks in 24-hour scenarios; however, significant challenges persist in achieving satisfactory performance due to the substantial discrepancies between visible (VIS) and infrared (IR) modalities. Existing methods inadequately leverage information from different modalities, primarily focusing on digging distinguishing features from modality-shared information while neglecting modality-specific details. To fully utilize differentiated minutiae, we propose a Base-Detail Feature Learning Framework (BDLF) that enhances the learning of both base and detail knowledge, thereby capitalizing on both modality-shared and modality-specific information. Specifically, the proposed BDLF mines detail and base features through a lossless detail feature extraction module and a complementary base embedding generation mechanism, respectively, supported by a novel correlation restriction method that ensures the features gained by BDLF enrich both detail and base knowledge across VIS and IR features. Comprehensive experiments conducted on the SYSU-MM01, RegDB, and LLCM datasets validate the effectiveness of BDLF.
Paper Structure (20 sections, 22 equations, 5 figures, 3 tables)

This paper contains 20 sections, 22 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Overall Framework
  5. Specific-shared Knowledge Distillation
  6. Detail Feature Extraction
  7. Base Embedding Generation
  8. Optimization
  9. Experiments
  10. Datasets and Evaluation Protocol
  11. Implementation Details
  12. Comparison with State-of-the-art Methods
  13. Comparison on SYSU-MM01 and RegDB.
  14. Comparison on LLCM.
  15. Ablation Studies
  16. ...and 5 more sections

Figures (5)

  • Figure 1: Motivation of the proposed BDLF, which focuses on sufficiently mining the modality-shared and modality-specific knowledge simultaneously and are not applicable for additional auxiliary data.
  • Figure 2: The pipeline of the proposed Base-Detail Feature Learning Framework (BDLF), which consists of a Detail Feature Extraction (DFE) module and a Base Embedding Generation (BEG) block, and jointly optimizes the extracted detail, base, and comprehensive features.
  • Figure 3: Illustration of correlation instruct to learn modality specific and shared information.
  • Figure 4: Effectiveness of how many INN blocks are more favorable for the proposed DFE.
  • Figure 5: Visualization of the comprehensive and detailed features.