ViTA-PAR: Visual and Textual Attribute Alignment with Attribute Prompting for Pedestrian Attribute Recognition

TL;DR

ViTA-PAR tackles pedestrian attribute recognition by aligning visual and textual attribute cues through dedicated visual prompts and a learnable person-and-attribute text prompt. The method learns a shared multimodal space via a visual-textual alignment objective, while inference relies solely on visual features for efficiency. Key contributions include visual attribute prompts, a novel contextual text prompting template, and a straightforward alignment mechanism that yields competitive PAR performance with faster inference. The approach demonstrates robust handling of coarse-to-fine attributes across diverse locations, with strong results on four benchmarks and practical deployment potential due to reduced computational cost.

Abstract

The Pedestrian Attribute Recognition (PAR) task aims to identify various detailed attributes of an individual, such as clothing, accessories, and gender. To enhance PAR performance, a model must capture features ranging from coarse-grained global attributes (e.g., for identifying gender) to fine-grained local details (e.g., for recognizing accessories) that may appear in diverse regions. Recent research suggests that body part representation can enhance the model's robustness and accuracy, but these methods are often restricted to attribute classes within fixed horizontal regions, leading to degraded performance when attributes appear in varying or unexpected body locations. In this paper, we propose Visual and Textual Attribute Alignment with Attribute Prompting for Pedestrian Attribute Recognition, dubbed as ViTA-PAR, to enhance attribute recognition through specialized multimodal prompting and vision-language alignment. We introduce visual attribute prompts that capture global-to-local semantics, enabling diverse attribute representations. To enrich textual embeddings, we design a learnable prompt template, termed person and attribute context prompting, to learn person and attributes context. Finally, we align visual and textual attribute features for effective fusion. ViTA-PAR is validated on four PAR benchmarks, achieving competitive performance with efficient inference. We release our code and model at https://github.com/mlnjeongpark/ViTA-PAR.

Figures (3)

• Figure 1: Comparison of ViTA-PAR with other visual and language-based PAR (VL-PAR) approaches: VTB vtb and PromptPAR wang2023promptPAR. (a) VTB uses ViT and BERT as feature extractors, resulting in separate representation spaces that hinder direct modality fusion. Moreover, the class token, trained with long-range attention, struggles to capture fine-grained attributes, as shown in the CAM visualization of "Hat". (b) PromptPAR leverages CLIP’s shared representation space and optimizes region-aware prompts by horizontally dividing image patches. However, due to the diverse attribute locations, the region prompt misaligns with textual embeddings, leading to suboptimal "Hat" predictions. (c) To address these issues, we propose visual and textual attribute prompts with visual-textual alignment. Our approach effectively captures "hat" despite its dynamic location, as evidenced by CAM results and a higher confidence score than (a) and (b).
• Figure 2: ViTA-PAR Architecture. ViTA-PAR introduces visual and textual attribute prompts to capture coarse-to-fine attribute cues and align them within a shared embedding space through visual-textual attribute alignment. Note that, during testing, only the image is fed into the visual encoder, excluding textual features, which leads to reduced computational cost.
• Figure 3: Class activation maps for randomly selected images from (a) PA-100K, (b) PETA, (c) RAPv1, and (d) RAPv2 datasets.