Modality-Transition Representation Learning for Visible-Infrared Person Re-Identification

TL;DR

The paper tackles the cross-modal VI-ReID challenge by bridging visible and infrared domains with a modality-transition representation learned during training. It introduces MTRL, which leverages a middle generated transition image as a bridge and enforces cross-modal alignment through $\mathcal{L}_{mtc}$, $\mathcal{L}_{center}$, and $\mathcal{L}_{mqr}$ without adding inference-time parameters. The approach achieves state-of-the-art performance on SYSU-MM01, RegDB, and LLCM across multiple settings, demonstrating strong, consistent gains in Rank-1 and mAP while maintaining backbone efficiency. By providing a parameter-efficient, training-time-only framework with interpretable distance-based losses, the work offers a practical path toward robust VI-ReID in real-world, illumination-variant scenarios.

Abstract

Visible-infrared person re-identification (VI-ReID) technique could associate the pedestrian images across visible and infrared modalities in the practical scenarios of background illumination changes. However, a substantial gap inherently exists between these two modalities. Besides, existing methods primarily rely on intermediate representations to align cross-modal features of the same person. The intermediate feature representations are usually create by generating intermediate images (kind of data enhancement), or fusing intermediate features (more parameters, lack of interpretability), and they do not make good use of the intermediate features. Thus, we propose a novel VI-ReID framework via Modality-Transition Representation Learning (MTRL) with a middle generated image as a transmitter from visible to infrared modals, which are fully aligned with the original visible images and similar to the infrared modality. After that, using a modality-transition contrastive loss and a modality-query regularization loss for training, which could align the cross-modal features more effectively. Notably, our proposed framework does not need any additional parameters, which achieves the same inference speed to the backbone while improving its performance on VI-ReID task. Extensive experimental results illustrate that our model significantly and consistently outperforms existing SOTAs on three typical VI-ReID datasets.

Figures (11)

• Figure 1: The problems of the existing methods and our main idea of our modality-transition representation learning. XIVli2020infrared and MIDHuang2022ModalityAdaptiveMA are some of works use generated images as intermediate, which participate training.
• Figure 2: Overview of the proposed MTRL framework for the VI-ReID task, including ➀ Modality-Aware Hierarchical Constraints (generative modality-transition contrastive loss $\mathcal{L}_{mtc}$ and global ID center loss $\mathcal{L}_{center}$), and ➁ Modality-Query Regularization $\mathcal{L}_{mqr}$.
• Figure 3: Sensitive graph of $\alpha$ on SYSU-MM01 with All-Search and Single-Shot settings, and RegDB with I2V mode.
• Figure 4: Sensitive graph of $\beta$ (left) on RegDB with I2V mode and $\gamma$ (right) on SYSU-MM01 with All-Search and Single-Shot mode.
• Figure 5: The impact of the number of top-k on RegDB with I2V mode. $N$ is the number of positive samples.