Robust Duality Learning for Unsupervised Visible-Infrared Person Re-Identification

TL;DR

A novel robust duality learning framework (RoDE) for UVI-ReID is proposed to mitigate the adverse impact of noisy pseudo-labels and a novel Robust Adaptive Learning mechanism (RAL) is proposed to dynamically prioritize clean samples while deprioritizing noisy ones, thus avoiding overemphasizing noise.

Abstract

Unsupervised visible-infrared person re-identification (UVI-ReID) aims to retrieve pedestrian images across different modalities without costly annotations, but faces challenges due to the modality gap and lack of supervision. Existing methods often adopt self-training with clustering-generated pseudo-labels but implicitly assume these labels are always correct. In practice, however, this assumption fails due to inevitable pseudo-label noise, which hinders model learning. To address this, we introduce a new learning paradigm that explicitly considers Pseudo-Label Noise (PLN), characterized by three key challenges: noise overfitting, error accumulation, and noisy cluster correspondence. To this end, we propose a novel Robust Duality Learning framework (RoDE) for UVI-ReID to mitigate the effects of noisy pseudo-labels. First, to combat noise overfitting, a Robust Adaptive Learning mechanism (RAL) is proposed to dynamically emphasize clean samples while down-weighting noisy ones. Second, to alleviate error accumulation-where the model reinforces its own mistakes-RoDE employs dual distinct models that are alternately trained using pseudo-labels from each other, encouraging diversity and preventing collapse. However, this dual-model strategy introduces misalignment between clusters across models and modalities, creating noisy cluster correspondence. To resolve this, we introduce Cluster Consistency Matching (CCM), which aligns clusters across models and modalities by measuring cross-cluster similarity. Extensive experiments on three benchmarks demonstrate the effectiveness of RoDE.

Figures (9)

• Figure 1: Pseudo-label noise issues in UVI-ReID. (a) In intra-modality, some sample features are close to the adjacent cluster center, leading to false pseudo-label assignments and noise overfitting. (b) Error accumulation for a single model (TOP) and dual models (BOTTOM) is depicted through the per-sample loss distribution on the infrared modality of RegDB dataset using the recent IMSL method pang2024inter. The dual models employ a cross-training strategy, using the pseudo-labels generated by one model as the ground truth for the other. During training, inevitable error annotations and cluster mismatches introduce significant noise. For a single model, noisy and clean samples intermingle due to severe error accumulation, as indicated by the overlapping color parts. In contrast, using dual models significantly mitigates this issue. (c) Semantic misalignment occurs across different clusters, including distinct models and modalities, which are regarded as noisy cluster correspondences.
• Figure 2: The framework of the proposed RoDE. The model projects the visible and infrared images into the common space using the modality-specific networks $f^{\mathcal{P}}(\cdot; \Theta^{\mathcal{P}})$. CCM (See \ref{['subsec:methodology5']}) and RAL (See \ref{['subsec:methodology3']}) are used to alleviate noisy cluster correspondence and noisy overfitting. Specifically, cross-modal and cross-model CCM are utilized to establish the correspondence across different modalities and different models, respectively. Moreover, RAL divides the pseudo-labels into clean and noisy subsets, and adaptively adjusts the focus on them, thereby enhancing robustness against noisy overfitting.
• Figure 3: The training pipeline of the proposed RoDE. RoDE consists of two individual models $A$ and $B$, which are trained collaboratively by exchanging their pseudo supervisions. Before training, RoDE pre-warms up the models $A$ and $B$ individually by predicting pseudo-labels and self-training. After warming up, the two models are co-trained with CCM and RAL.
• Figure 4: The solution of cluster inconsistency issue. and represent visible and infrared modality centers respectively. The green dotted lines denote correct matches after CCM.
• Figure 5: The impact of parameter $\lambda$. The gray shaded area represents the recommended parameter range for further fine-tuning, as suggested by the authors.