Identity-aware Dual-constraint Network for Cloth-Changing Person Re-identification

TL;DR

A Clothes Diversity Augmentation (CDA) is proposed, which generates more realistic cloth-changing samples by enriching the clothing color while preserving the texture, and a Counterfactual-guided Attention Module (CAM) is presented, which learns cloth-irrelevant features from channel and space dimensions and utilizes the counterfactual intervention for supervising the attention map to highlight identity-related regions.

Abstract

Cloth-Changing Person Re-Identification (CC-ReID) aims to accurately identify the target person in more realistic surveillance scenarios, where pedestrians usually change their clothing. Despite great progress, limited cloth-changing training samples in existing CC-ReID datasets still prevent the model from adequately learning cloth-irrelevant features. In addition, due to the absence of explicit supervision to keep the model constantly focused on cloth-irrelevant areas, existing methods are still hampered by the disruption of clothing variations. To solve the above issues, we propose an Identity-aware Dual-constraint Network (IDNet) for the CC-ReID task. Specifically, to help the model extract cloth-irrelevant clues, we propose a Clothes Diversity Augmentation (CDA), which generates more realistic cloth-changing samples by enriching the clothing color while preserving the texture. In addition, a Multi-scale Constraint Block (MCB) is designed, which extracts fine-grained identity-related features and effectively transfers cloth-irrelevant knowledge. Moreover, a Counterfactual-guided Attention Module (CAM) is presented, which learns cloth-irrelevant features from channel and space dimensions and utilizes the counterfactual intervention for supervising the attention map to highlight identity-related regions. Finally, a Semantic Alignment Constraint (SAC) is designed to facilitate high-level semantic feature interaction. Comprehensive experiments on four CC-ReID datasets indicate that our method outperforms prior state-of-the-art approaches.

Figures (10)

• Figure 1: A schematic diagram of our approach. Data augmentation can effectively improve the variety of clothing and generate more realistic cloth-changing samples. The attention module can motivate the network to concentrate on areas not related to clothing. Dual constraints can realize cloth-agnostic knowledge transfer at both shallow and deep layers of the model.
• Figure 2: (a) Overall framework of our proposed identity-aware dual-constraint network (IDNet). The top is the raw image stream, while the bottom is the clothing erasing stream. After performing Clothes Diversity Augmentation (CDA) on the training set, the augmented images and black-clothing images are input into backbone networks with the shared-weight architecture. Between stages of the backbone network, the proposed Multi-scale Constraint Blocks (MCBs) are inserted to extract multi-scale features and transfer cloth-independent knowledge to the raw image stream. After the backbone network, the Counterfactual-guided Attention Module (CAM) highlights identity-related features and utilizes counterfactual intervention to optimize the quality of learned attention maps. Finally, the Semantic Alignment Constraint (SAC) facilitates the raw image stream to learn cloth-agnostic semantic features. Moreover, each stream is supervised by triplet loss $L_{tri}$ and identity loss $L_{id}$. (b) Illustration of our proposed MCB. (c) Illustration of our proposed CAM.
• Figure 3: Comparison of the proposed clothes diversity augmentation (CDA) and domain augmentation (DA) chen2023unsupervised. Each column represents a specific order of channels.
• Figure 4: Elaboration of causal relationship and counterfactual intervention in the causal graph.
• Figure 5: Illustration of the Semantic Alignment Constraint (SAC).