Disentangled Concept Representation for Text-to-image Person Re-identification

TL;DR

TIReID faces a modality gap and the need for fine-grained cross-modal grounding. The authors introduce DiCo, a Disentangled Concept Representation that uses a shared K-slot bank with M concept blocks to enable hierarchical, interpretable alignment across global, part, and concept levels between images and text, learned with iterative slot-concept refinement and shared prototype memories. The model employs multi-level losses, including global, slot, block, identity, and reconstruction terms, and uses a CLIP-based backbone to achieve competitive results while providing interpretable slot and block-level representations. Experiments on CUHK-PEDES, ICFG-PEDES, and RSTPReid show strong performance and clear qualitative evidence of disentangled concept grounding, demonstrating practical improvements in fine-grained TIReID without explicit part annotations.

Abstract

Text-to-image person re-identification (TIReID) aims to retrieve person images from a large gallery given free-form textual descriptions. TIReID is challenging due to the substantial modality gap between visual appearances and textual expressions, as well as the need to model fine-grained correspondences that distinguish individuals with similar attributes such as clothing color, texture, or outfit style. To address these issues, we propose DiCo (Disentangled Concept Representation), a novel framework that achieves hierarchical and disentangled cross-modal alignment. DiCo introduces a shared slot-based representation, where each slot acts as a part-level anchor across modalities and is further decomposed into multiple concept blocks. This design enables the disentanglement of complementary attributes (\textit{e.g.}, color, texture, shape) while maintaining consistent part-level correspondence between image and text. Extensive experiments on CUHK-PEDES, ICFG-PEDES, and RSTPReid demonstrate that our framework achieves competitive performance with state-of-the-art methods, while also enhancing interpretability through explicit slot- and block-level representations for more fine-grained retrieval results.

Figures (8)

• Figure 1: Illustration of alignment strategies for TIReID. (a) Global matching: aligns image and text only at the global level, missing local details. (b) Part-level matching: introduces region–word alignment but still entangles multiple attributes within each part. (c) Concept-level matching (Ours): disentangles slots into concept-specific blocks, enabling hierarchical alignment from global identity cues to fine-grained attributes such as color, texture, and shape.
• Figure 2: (a) Overall framework: Given an input image and text, visual and textual features are extracted by respective encoders, followed by global alignment and refinement through shared slots. The refined slot representations are aligned at both part- and concept-levels to capture fine-grained cross-modal correspondences. (b) Slot-concept Attention: disentangles slot representations into interpretable concept blocks while ensuring semantic consistency across modalities.
• Figure 3: Qualitative results on CUHK-PEDES CUHK-PEDES. Each query description (left) is shown with the top-5 retrieved images (right). Green boxes indicate correct matches.
• Figure 4: Visualization of slot-level attention for text queries and corresponding images. Each slot attends to distinct body regions and aligns with relevant words in the query, effectively capturing fine-grained attributes such as color, clothing type, and shoes.
• Figure 5: t-SNE JMLR:v9:vandermaaten08a visualization of concept block embeddings. We project the learned concept representations into a 2D space using t-SNE to examine whether different blocks capture distinct semantic subspaces. Each scatter plot represents a different sampled subset of concept embeddings, where points are colored by their corresponding block index. Across multiple views, we observe consistently separated clusters among the eight blocks, indicating that DiCo learns specialized and semantically diverse concept representations rather than collapsing into overlapping embedding regions.