Progressive Feature Learning for Realistic Cloth-Changing Gait Recognition

TL;DR

A new framework called Progressive Feature Learning is proposed that can be applied with off-the-shelf backbones to improve recognition performance, especially in the cloth-changing conditions and can effectively improve recognition performance, especially in the cloth-changing conditions.

Abstract

Gait recognition is instrumental in crime prevention and social security, for it can be conducted at a long distance to figure out the identity of persons. However, existing datasets and methods cannot satisfactorily deal with the most challenging cloth-changing problem in practice. Specifically, the practical gait models are usually trained on automatically labeled data, in which the sequences' views and cloth conditions of each person have some restrictions. To be concrete, the cross-view sub-dataset only has normal walking condition without cloth-changing, while the cross-cloth sub-dataset has cloth-changing sequences but only in front views. As a result, the cloth-changing accuracy cannot meet practical requirements. In this work, we formulate the problem as Realistic Cloth-Changing Gait Recognition (abbreviated as RCC-GR) and we construct two benchmarks: CASIA-BN-RCC and OUMVLP-RCC, to simulate the above setting. Furthermore, we propose a new framework called Progressive Feature Learning that can be applied with off-the-shelf backbones to improve their performance in RCC-GR. Specifically, in our framework, we design Progressive Mapping and Progressive Uncertainty to extract cross-view features and then extract cross-cloth features on the basis. In this way, the feature from the cross-view sub-dataset can first dominate the feature space and relieve the uneven distribution caused by the adverse effect from the cross-cloth sub-dataset. The experiments on our benchmarks show that our framework can effectively improve recognition performance, especially in the cloth-changing conditions.

Figures (5)

• Figure 1: The automatic label pipeline. The images are come from DeepChange dataset xu2021long.
• Figure 2: The composition of a real cloth-changing benchmark. (a) Cross-view sub-dataset. Each person has view variations but only has the normal walking condition (NM). (b) Cross-cloth sub-dataset. Each identity has walking in different coats condition (CL) but only has limited views (only front views).
• Figure 3: The structure of our framework. Backbone is used to extract features from silhouettes. The feature $f$ outputs from the backbone will be put into two branches, and each uses Progressive Mapping with Cross-View Mapping module (CVM) and Cross-Cloth Mapping module (CCM) to generate $\mu_{j,i}/\sigma_{j,i};\ j \in v,c; \ i \in 1,2,...,S$. Head is a light block used to isolate features of two branches. Progressive Uncertainty constructs two Gaussian distributions with learned mean and variance features to relax constraints on feature mapping.
• Figure 4: The comparision of sequence number of each cloth condition between different datasets.
• Figure 5: Some examples of the generated cloth-changing images. The first line shows the original images, the second line shows the generated image after dilation.