Cross-Covariate Gait Recognition: A Benchmark

TL;DR

The paper presents CCGR, the first large-scale, well-annotated gait recognition benchmark explicitly designed to study cross-covariate effects, with 970 subjects, ~1.6M sequences, 53 covariates and 33 views across multiple data modalities. It introduces ParsingGait, a parsing-based baseline that leverages semantic parsing maps to enrich gait representations while remaining compatible with existing silhouette-based methods. Experimental results show cross-covariate variation markedly degrades performance, with state-of-the-art methods achieving less than 43% rank-1 accuracy, and ParsingGait delivering notable gains by exploiting parsing information. The work highlights the critical role of both population- and individual-level diversity and establishes ParsingGait as a promising direction for robust gait recognition in real-world, covariate-rich environments, providing a rich resource for researchers and practitioners in the field.

Abstract

Gait datasets are essential for gait research. However, this paper observes that present benchmarks, whether conventional constrained or emerging real-world datasets, fall short regarding covariate diversity. To bridge this gap, we undertake an arduous 20-month effort to collect a cross-covariate gait recognition (CCGR) dataset. The CCGR dataset has 970 subjects and about 1.6 million sequences; almost every subject has 33 views and 53 different covariates. Compared to existing datasets, CCGR has both population and individual-level diversity. In addition, the views and covariates are well labeled, enabling the analysis of the effects of different factors. CCGR provides multiple types of gait data, including RGB, parsing, silhouette, and pose, offering researchers a comprehensive resource for exploration. In order to delve deeper into addressing cross-covariate gait recognition, we propose parsing-based gait recognition (ParsingGait) by utilizing the newly proposed parsing data. We have conducted extensive experiments. Our main results show: 1) Cross-covariate emerges as a pivotal challenge for practical applications of gait recognition. 2) ParsingGait demonstrates remarkable potential for further advancement. 3) Alarmingly, existing SOTA methods achieve less than 43% accuracy on the CCGR, highlighting the urgency of exploring cross-covariate gait recognition. Link: https://github.com/ShinanZou/CCGR.

Figures (11)

• Figure 1: Differences between CCGR and other datasets. Population-level diversity is roughly quantified by the count of covariate categories present within the whole dataset. Correspondingly, individual-level diversity is measured by the count of covariate categories for each subject. Here, the population-level diversity of Gait3D and GREW is rich, but the exact amount is unknown due to the wild scenarios.
• Figure 2: Examples of 53 covariates in CCGR. For a single covariate (the 1st row and the left of the 2nd row), the red numbers at the top of the pictures are indices of the covariates. For mixed covariates, numbers separated by "/" at the top of the picture indicate the co-occur of multi-single covariates corresponding to these numbers.
• Figure 3: Examples of 33 views in CCGR. The red numbers at the top of the picture represent the horizontal angle.
• Figure 4: Camera setup in CCGR.
• Figure 5: Examples of different gait data in CCGR .