Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Content and Salient Semantics Collaboration for Cloth-Changing Person Re-Identification

Qizao Wang, Xuelin Qian, Bin Li, Lifeng Chen, Yanwei Fu, Xiangyang Xue

TL;DR

This work tackles cloth-changing person Re-ID by eliminating reliance on auxiliary modalities and annotations. It introduces the Semantics Mining and Refinement (SMR) module to extract robust content semantics via average pooling and local regions, and salient semantics via max pooling, followed by channel-wise refinement; these semantics are then combined in the Content and Salient Semantics Collaboration (CSSC) framework that enables cross-parallel interaction to reinforce identity cues despite clothing changes. The model is trained with joint losses on global and local semantics, followed by a cross-branch collaboration that fuses refined content and salient semantics into a final RGB-only representation $f^{cssc}$ optimized with standard Re-ID losses. Experiments on PRCC, LTCC, and Celeb-reID show state-of-the-art or competitive performance without extra annotations or auxiliary data, highlighting the practicality and effectiveness of leveraging abundant semantics within pedestrian images for cloth-changing Re-ID.

Abstract

Cloth-changing person re-identification aims at recognizing the same person with clothing changes across non-overlapping cameras. Advanced methods either resort to identity-related auxiliary modalities (e.g., sketches, silhouettes, and keypoints) or clothing labels to mitigate the impact of clothes. However, relying on unpractical and inflexible auxiliary modalities or annotations limits their real-world applicability. In this paper, we promote cloth-changing person re-identification by leveraging abundant semantics present within pedestrian images, without the need for any auxiliaries. Specifically, we first propose a unified Semantics Mining and Refinement (SMR) module to extract robust identity-related content and salient semantics, mitigating interference from clothing appearances effectively. We further propose the Content and Salient Semantics Collaboration (CSSC) framework to collaborate and leverage various semantics, facilitating cross-parallel semantic interaction and refinement. Our proposed method achieves state-of-the-art performance on three cloth-changing benchmarks, demonstrating its superiority over advanced competitors. The code is available at https://github.com/QizaoWang/CSSC-CCReID.

Content and Salient Semantics Collaboration for Cloth-Changing Person Re-Identification

TL;DR

This work tackles cloth-changing person Re-ID by eliminating reliance on auxiliary modalities and annotations. It introduces the Semantics Mining and Refinement (SMR) module to extract robust content semantics via average pooling and local regions, and salient semantics via max pooling, followed by channel-wise refinement; these semantics are then combined in the Content and Salient Semantics Collaboration (CSSC) framework that enables cross-parallel interaction to reinforce identity cues despite clothing changes. The model is trained with joint losses on global and local semantics, followed by a cross-branch collaboration that fuses refined content and salient semantics into a final RGB-only representation optimized with standard Re-ID losses. Experiments on PRCC, LTCC, and Celeb-reID show state-of-the-art or competitive performance without extra annotations or auxiliary data, highlighting the practicality and effectiveness of leveraging abundant semantics within pedestrian images for cloth-changing Re-ID.

Abstract

Cloth-changing person re-identification aims at recognizing the same person with clothing changes across non-overlapping cameras. Advanced methods either resort to identity-related auxiliary modalities (e.g., sketches, silhouettes, and keypoints) or clothing labels to mitigate the impact of clothes. However, relying on unpractical and inflexible auxiliary modalities or annotations limits their real-world applicability. In this paper, we promote cloth-changing person re-identification by leveraging abundant semantics present within pedestrian images, without the need for any auxiliaries. Specifically, we first propose a unified Semantics Mining and Refinement (SMR) module to extract robust identity-related content and salient semantics, mitigating interference from clothing appearances effectively. We further propose the Content and Salient Semantics Collaboration (CSSC) framework to collaborate and leverage various semantics, facilitating cross-parallel semantic interaction and refinement. Our proposed method achieves state-of-the-art performance on three cloth-changing benchmarks, demonstrating its superiority over advanced competitors. The code is available at https://github.com/QizaoWang/CSSC-CCReID.
Paper Structure (11 sections, 10 equations, 3 figures, 4 tables)

This paper contains 11 sections, 10 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Problem Formulation
  4. Semantics Mining and Refinement Module
  5. Semantics Collaboration Framework
  6. Training and Inference
  7. Experiments
  8. Experimental Settings
  9. Comparison with State-of-the-Art Methods
  10. Ablation Studies
  11. Conclusion

Figures (3)

  • Figure 1: Comparison of applying different poolings on pedestrian images. Average pooling can better preserve image content while smoothing details, while max pooling is better at capturing salient information in the content. Incorporating both of them is expected to learn abundant semantics, and thus improve the discriminative ability of Re-ID models.
  • Figure 2: Framework of our method. The Semantics Mining and Refinement (SMR) module learns identity-related semantics without relying on any auxiliaries. SMR modules equipped with average pooling (SMR-C) and max pooling (SMR-S) learn content and salient semantics effectively, respectively. Taking advantage of SMR, our framework interacts and refines both semantics sequentially and parallelly to promote cloth-changing person Re-ID.
  • Figure 3: Visualization of top-10 retrieval results. For each query image, the first and the second rows are the ordered matching results obtained by using the baseline ResNet-50 and our proposed CSSC, respectively. Images with green and red borders indicate correct and wrong matching results, respectively. The results are obtained in the cloth-changing setting on LTCC.