Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Language-Guided and Motion-Aware Gait Representation for Generalizable Recognition

Zhengxian Wu, Chuanrui Zhang, Shenao Jiang, Hangrui Xu, Zirui Liao, Luyuan Zhang, Huaqiu Li, Peng Jiao, Haoqian Wang

TL;DR

This work addresses the noise-prone nature of RGB gait recognition by introducing LMGait, a framework that injects domain-specific language descriptions as semantic priors to guide visual features toward motion-relevant regions. A Motion Awareness Module (MAM) refines text representations for better cross-modal alignment, while a Motion Temporal Capture Module (MTCM) jointly models pixel- and region-level gait dynamics under language guidance. The method combines a frozen image encoder (Dinov2), a training-time text branch, a representation extractor with a multi-loss objective, and a downstream gait classifier, achieving state-of-the-art results on CCPG, SUSTech1K, and CASIAB* and offering efficient inference by caching language-guided attention. Ablation studies confirm the complementary roles of MAM and MTCM, and demonstrate robustness to text quality and encoder choices, underscoring the practical impact of language-guided gait representations for generalizable recognition.

Abstract

Gait recognition is emerging as a promising technology and an innovative field within computer vision. However, existing methods typically rely on complex architectures to directly extract features from images and apply pooling operations to obtain sequence-level representations. Such designs often lead to overfitting on static noise (e.g., clothing), while failing to effectively capture dynamic motion regions.To address the above challenges, we present a Language guided and Motion-aware gait recognition framework, named LMGait.In particular, we utilize designed gait-related language cues to capture key motion features in gait sequences.

Language-Guided and Motion-Aware Gait Representation for Generalizable Recognition

TL;DR

This work addresses the noise-prone nature of RGB gait recognition by introducing LMGait, a framework that injects domain-specific language descriptions as semantic priors to guide visual features toward motion-relevant regions. A Motion Awareness Module (MAM) refines text representations for better cross-modal alignment, while a Motion Temporal Capture Module (MTCM) jointly models pixel- and region-level gait dynamics under language guidance. The method combines a frozen image encoder (Dinov2), a training-time text branch, a representation extractor with a multi-loss objective, and a downstream gait classifier, achieving state-of-the-art results on CCPG, SUSTech1K, and CASIAB* and offering efficient inference by caching language-guided attention. Ablation studies confirm the complementary roles of MAM and MTCM, and demonstrate robustness to text quality and encoder choices, underscoring the practical impact of language-guided gait representations for generalizable recognition.

Abstract

Gait recognition is emerging as a promising technology and an innovative field within computer vision. However, existing methods typically rely on complex architectures to directly extract features from images and apply pooling operations to obtain sequence-level representations. Such designs often lead to overfitting on static noise (e.g., clothing), while failing to effectively capture dynamic motion regions.To address the above challenges, we present a Language guided and Motion-aware gait recognition framework, named LMGait.In particular, we utilize designed gait-related language cues to capture key motion features in gait sequences.
Paper Structure (15 sections, 18 equations, 5 figures, 7 tables)

This paper contains 15 sections, 18 equations, 5 figures, 7 tables.

Table of Contents

  1. 1 Introduction
  2. 2 Related Work
  3. 2.1 Gait Recognition
  4. 2.2 Fine-tuning Text Encoder for Improved Visual Alignment
  5. 3 Method
  6. 3.1 Pipeline
  7. 3.2 Text Encoder
  8. 3.3 Motion Temporal Capture Module
  9. 3.4 Representation Extractor and Loss Function
  10. 4 Experiments
  11. 4.1 Experimental Setup
  12. 4.2 Comparison with State-Of-The-Art Methods
  13. 4.3 Ablation Study and Visualization
  14. 5 Conclusion
  15. 6 Acknowledgments

Figures (5)

  • Figure 1: The Rank-1 accuracies of several networks on the CCPG CCPG and SUSTech1K SUSTech1K datasets are presented, involving three representations: Silhouette, Skeleton, and RGB. The results for CCPG are shown in red, while those for SUSTech1K are shown in blue.
  • Figure 2: Pipeline of the proposed LMGait, it consists of five components.Specifically, the video input is processed through the frozen Dinov2 model for feature extraction. The text query guides the network to focus on gait-relevant regions, and it is aligned with the image feature space through the frozen CLIP text encoder and the fine-tuned MAM module. The Representation Extractor generates diverse features, while the Motion Temporal Capture Module captures posture changes during walking. Finally, the extracted features are input into the Gait Network for recognition.
  • Figure 3: The architecture of the Text Encoder is designed to improve the alignment between text and image features. To achieve this, we introduce the MAM module and fine-tune it.
  • Figure 4: The architecture of the Motion Temporal Capture Module operates from two complementary perspectives: pixel-wise temporal extraction and region-wise temporal extraction. The two branches collaboratively capture gait information.
  • Figure 5: Visualization of attention maps from the first layer of the downstream gait recognition network.