Clothes-Changing Person Re-Identification with Feasibility-Aware Intermediary Matching

TL;DR

The paper tackles clothes-changing person re-identification by introducing FAIM, a framework that leverages clothes-relevant features to locate informative intermediaries and uses a feasibility-aware weighting scheme to handle varying intermediary quality. It combines four components—Feature Decoupling, Identity Information Reliability, Intermediary Matching with three routes, and Intermediary-Based Feasibility Weighting—to enable robust cross-clothes matching. Across LTCC, PRCC, and DeepChange benchmarks using RGB data, FAIM achieves state-of-the-art performance, demonstrating that integrating clothes-relevant cues via intermediary routing substantially improves identification under clothing changes. The approach has practical significance for surveillance systems while underscoring the need for privacy-preserving data practices.

Abstract

Current clothes-changing person re-identification (re-id) approaches usually perform retrieval based on clothes-irrelevant features, while neglecting the potential of clothes-relevant features. However, we observe that relying solely on clothes-irrelevant features for clothes-changing re-id is limited, since they often lack adequate identity information and suffer from large intra-class variations. On the contrary, clothes-relevant features can be used to discover same-clothes intermediaries that possess informative identity clues. Based on this observation, we propose a Feasibility-Aware Intermediary Matching (FAIM) framework to additionally utilize clothes-relevant features for retrieval. Firstly, an Intermediary Matching (IM) module is designed to perform an intermediary-assisted matching process. This process involves using clothes-relevant features to find informative intermediates, and then using clothes-irrelevant features of these intermediates to complete the matching. Secondly, in order to reduce the negative effect of low-quality intermediaries, an Intermediary-Based Feasibility Weighting (IBFW) module is designed to evaluate the feasibility of intermediary matching process by assessing the quality of intermediaries. Extensive experiments demonstrate that our method outperforms state-of-the-art methods on several widely-used clothes-changing re-id benchmarks.

Figures (7)

• Figure 1: Illustrative examples of intermediary matching approach. (A) for query lacking clothes-irrelevant identity information (facial representation), we can match it to target through an intermediary with clear facial information. (B) for the query and target suffering large intra-class variation (body shape), we can match them through an intermediary with aligned body shape. (C) and (D) represent intermediaries of query (B) with low availability and low reliability, respectively.
• Figure 2: (a) Illustrative examples of intermediary availability. When matching with clothes-relevant features, we consider the availability as the accessibility of same-clothes samples. When matching with clothes-irrelevant features, we consider the availability as the accessibility of same-identity samples. (b) Illustrative examples of intermediary reliability. High reliability samples usually have clear clothes-irrelevant cues (facial view and body shape), while low reliability samples typically lack integrity in face and body shape.
• Figure 3: Overview of our FAIM framework. (1) A Feature Decoupling module is utilized to extract clothes-relevant feature $\boldsymbol{f}^{re}$ and clothes-irrelevant feature $\boldsymbol{f}^{ir}$. (2) An Identity Information Reliability module is designed to predict reliability score of identity information in $\boldsymbol{f}^{ir}$. (3) An Intermediary Matching module is proposed, which conducts three matching routes $\mathcal{A},\mathcal{B},\mathcal{C}$ to comprehensively address situations where direct matching solely based on clothes-irrelevant features is ineffective. (4) An Intermediary-Based Feasibility Weighting module is utilized to assign feasibility weights to routes $\mathcal{A}\sim \mathcal{C}$ respectively, according to the availability and reliability of intermediaries.
• Figure 4: The top-1 accuracy and mAP with different $s_A$, $s_B$ and $s_C$ on the clothes-changing setting of LTCC. In (a) , we fix $s_B$ to $0.6$ and $s_C$ to $0.1$. In (b) , we fix $s_A$ to $0.3$ and $sa_C$ to $0.1$. In (c) , we fix $s_A$ to $0.3$ and $s_B$ to $0.6$.
• Figure 5: Visualization results of person with low and high reliability score of clothes-irrelevant identity information ($r^{id}$). The left side shows samples with $r^{id} < 0.5$, while the right side shows samples with $r^{id} > 0.5$.