MimicGait: A Model Agnostic approach for Occluded Gait Recognition using Correlational Knowledge Distillation

TL;DR

This work proposes MimicGail, a model-agnostic approach for gait recognition in the presence of occlusion, using a multi-instance correlational distillation loss to capture both inter-sequence and intra-sequence correlations in the occluded gait patterns of a subject.

Abstract

Gait recognition is an important biometric technique over large distances. State-of-the-art gait recognition systems perform very well in controlled environments at close range. Recently, there has been an increased interest in gait recognition in the wild prompted by the collection of outdoor, more challenging datasets containing variations in terms of illumination, pitch angles, and distances. An important problem in these environments is that of occlusion, where the subject is partially blocked from camera view. While important, this problem has received little attention. Thus, we propose MimicGait, a model-agnostic approach for gait recognition in the presence of occlusions. We train the network using a multi-instance correlational distillation loss to capture both inter-sequence and intra-sequence correlations in the occluded gait patterns of a subject, utilizing an auxiliary Visibility Estimation Network to guide the training of the proposed mimic network. We demonstrate the effectiveness of our approach on challenging real-world datasets like GREW, Gait3D and BRIAR. We release the code in https://github.com/Ayush-00/mimicgait.

Figures (8)

• Figure 1: Visualizations of the synthetic occlusions used in our experiments, taken from the GREW dataset. The original holistic video is shown in the top left. Middle occlusions are shown in the top right. The second row shows the same video with consistent synthetic occlusions. The bottom most row shows the same video with dynamic occlusions. The boundary of the moving occlusion patch is shown in red for visualization purposes only.
• Figure 2: Overview of the proposed approach. The training procedure consists of two stages. In the pretraining stage, the Visibility Estimation Network $\mathcal{V}$ (also called VEN) and the teacher network $\mathcal{F}_t$ are trained. In the distillation stage, a new mimic network $\mathcal{F}_m$ is trained by $\mathcal{F}_t$ using a multi-instance correlational KD loss. $\mathcal{V}$ is used to guide $\mathcal{F}_m$ by injecting occlusion-relevant features. The $i^{th}$ video of subject $a$ may be occluded (denoted by $O_i^a$) or holistic (denoted by $C_i^a$). Their representations in the latent space are denoted by $\gamma_m$ and $\gamma_t$ respectively. The three types of anchor-positive pairs sampled by the proposed loss seen in the figure are described in \ref{['sec:mickd-loss']}.
• Figure 3: Some samples images taken from the BRIAR dataset for two subjects. From left to right, the range of capture increases from close range to 1000m for each subject. A large variation in the quality of the videos and the collection conditions can be seen. Subjects have consented to the use of these images in publication.
• Figure 4: Comparing two hypothetical occlusion-mitigating methods M1/M2 between two backbones B1/B2 on occluded performance ($OP$) and holistic performance ($HP$). A small change $\Delta y_1$ in $OP$ can cause a large change in the slope/RP for B1, but a larger $\Delta y_2$ is needed to cause a similar change in slope for B2. By considering the slope rather than just the $OP$, RP is able to better isolate the effect of M1/M2 across backbones.
• Figure 5: Some more sample frames taken from videos present in the BRIAR dataset. Subjects have consented to use of these images. Each row consists of images of one subject. The two leftmost images in both rows show examples of the indoor, controlled gallery sequences. The remaining images are captured outdoors, and they make up the probe set. As the distance increases from left to right, the quality of the frames drops significantly.