Semantic-Aligned Learning with Collaborative Refinement for Unsupervised VI-ReID

TL;DR

This work addresses unsupervised VI-ReID by tackling cross-modality gaps that arise from fine-grained appearance differences. It introduces SALCR, a four-component framework comprising DAGL for bi-directional cross-modality label transfer via OT-based association, FGSAL for fine-grained semantic-aligned learning with query-guided part features, GPCR for online refinement of positives, and CMFP for efficient cross-modality feature propagation and re-ranking. The optimization combines $L = L_{dagl} + L_{fgsal} + \lambda L_{gpcr}$, and training alternates between an intra-modality clustering phase and the SALCR learning phase, with CMFP usable in both training and testing. Empirical results on SYSU-MM01 and RegDB show state-of-the-art performance among unsupervised VI-ReID methods, with notable gains from incorporating fine-grained, modality-specific semantics and online refinement, demonstrating practical impact for cross-modality person re-identification without annotations. The CMFP component adds efficient post-processing that further boosts retrieval performance while maintaining scalability.

Abstract

Unsupervised visible-infrared person re-identification (USL-VI-ReID) seeks to match pedestrian images of the same individual across different modalities without human annotations for model learning. Previous methods unify pseudo-labels of cross-modality images through label association algorithms and then design contrastive learning framework for global feature learning. However, these methods overlook the cross-modality variations in feature representation and pseudo-label distributions brought by fine-grained patterns. This insight results in insufficient modality-shared learning when only global features are optimized. To address this issue, we propose a Semantic-Aligned Learning with Collaborative Refinement (SALCR) framework, which builds up optimization objective for specific fine-grained patterns emphasized by each modality, thereby achieving complementary alignment between the label distributions of different modalities. Specifically, we first introduce a Dual Association with Global Learning (DAGI) module to unify the pseudo-labels of cross-modality instances in a bi-directional manner. Afterward, a Fine-Grained Semantic-Aligned Learning (FGSAL) module is carried out to explore part-level semantic-aligned patterns emphasized by each modality from cross-modality instances. Optimization objective is then formulated based on the semantic-aligned features and their corresponding label space. To alleviate the side-effects arising from noisy pseudo-labels, we propose a Global-Part Collaborative Refinement (GPCR) module to mine reliable positive sample sets for the global and part features dynamically and optimize the inter-instance relationships. Extensive experiments demonstrate the effectiveness of the proposed method, which achieves superior performances to state-of-the-art methods. Our code is available at \href{https://github.com/FranklinLingfeng/code-for-SALCR}.

Figures (12)

• Figure 1: Illustration of our idea. (a) illustrates the distinct feature distributions observed within different modalities, where features from different modalities lie in different subspaces. (b) showcases the inconsistency between pseudo-label distributions of instances within different modalities. (c) depicts our motivation: to optimize the fine-grained patterns emphasized by each modality through their respective label space.
• Figure 2: Overall framework of our proposed SALCR. Our framework mainly contains four components: (a) Dual Association with Global Learning (DAGL); (b) Fine-Grained Semantic-Aligned Learning (FGSAL); (c) Global-Part Collaborative Refinement (GPCR) and (d) Cross-Modality Feature Propagation (CMFP). The DAGL module generates modality-unified pseudo-labels at the beginning of each epoch. During training, the FGSAL and GPCR modules are executed. The FGSAL module first explores fine-grained semantic-aligned patterns and then optimizes them through cluster-level memories. The GPCR module mines reliable positive samples for global and part features from instance memories. The CMFP module further enhances the association and retrieval as pre-processing and post-processing steps.
• Figure 3: Illustration of (a) the instance-adaptive query generation module and (b) query-guided attention.
• Figure 4: Illustration of the cross-modality intersection strategy (a) for global features and the mutual correction strategy (b) for part features. Circles with letters represent k-nearest neighbors of the input features. Different letters indicate different instance indexes in instance memory banks.
• Figure 5: Parameter analysis of $\lambda$ and $k$ for GPCR on SYSU-MM01 dataset.