Learning Geometric Invariance for Gait Recognition

TL;DR

This work tackles the challenge of identity-invariant gait recognition under diverse conditions by reframing condition variations as geometric transformations. It introduces the $\mathcal{R}$eflect-$\mathcal{R}$otate-$\mathcal{S}$cale invariance learning framework, $\mathcal{RRS}$-Gait, which consists of three modules that enforce reflect, rotate, and scale equivariance (ReEL, RoEL, SEL) and then pool these features into a robust invariant representation. Through extensive experiments on four challenging datasets, including Gait3D, GREW, CCPG, and SUSTech1K, the approach yields consistent improvements in rank-1 accuracy and mAP, demonstrating strong cross-condition generalization. The work offers a principled pathway to leverage geometric priors for gait recognition, potentially enabling better performance under unseen views, clothing, and poses, with practical impact on surveillance and identity verification systems.

Abstract

The goal of gait recognition is to extract identity-invariant features of an individual under various gait conditions, e.g., cross-view and cross-clothing. Most gait models strive to implicitly learn the common traits across different gait conditions in a data-driven manner to pull different gait conditions closer for recognition. However, relatively few studies have explicitly explored the inherent relations between different gait conditions. For this purpose, we attempt to establish connections among different gait conditions and propose a new perspective to achieve gait recognition: variations in different gait conditions can be approximately viewed as a combination of geometric transformations. In this case, all we need is to determine the types of geometric transformations and achieve geometric invariance, then identity invariance naturally follows. As an initial attempt, we explore three common geometric transformations (i.e., Reflect, Rotate, and Scale) and design a $\mathcal{R}$eflect-$\mathcal{R}$otate-$\mathcal{S}$cale invariance learning framework, named ${\mathcal{RRS}}$-Gait. Specifically, it first flexibly adjusts the convolution kernel based on the specific geometric transformations to achieve approximate feature equivariance. Then these three equivariant-aware features are respectively fed into a global pooling operation for final invariance-aware learning. Extensive experiments on four popular gait datasets (Gait3D, GREW, CCPG, SUSTech1K) show superior performance across various gait conditions.

Figures (5)

• Figure 1: Different gait conditions are approximately associated with the combination of various geometric transformations, e.g., reflect, rotate, scale. Achieving invariance learning among them is an effective way to achieve identity invariance in gait recognition.
• Figure 2: An example weiler2023EquivariantAndCoordinateIndependentCNNs illustrates equivariance in convolution. When applying a reflection transformation to the input image, the regular convolution kernel will produce non-equivariant feature maps, i.e., $\mathcal{F}(\mathcal{T}(\mathcal{X}))\neq\mathcal{T}(\mathcal{F}(\mathcal{X}))$. However, by introducing both regular kernel and its reflect kernel for feature extraction, and applying max operation along their corresponding channels, we can obtain equivariant feature maps, i.e., $\mathcal{F}(\mathcal{T}(\mathcal{X}))=\mathcal{T}(\mathcal{F}(\mathcal{X}))$.
• Figure 3: $\mathcal{R}$eflect-$\mathcal{R}$otate-$\mathcal{S}$cale invariance learning framework ($\mathcal{RRS}$-Gait) involves three equivariance learning modules and corresponding invariant pooling layers: (a) Reflect Equivariance Learning (ReEL) module, (b) Adaptive Rotate Equivariance Learning (RoEL) module, and (c) Multi-Scale Equivariance Learning (SEL) module respectively introduce reflect kernels, rotate kernels, and multiple dilated kernels to achieve feature equivariance or approximate equivariance. These equivariant features are then fed into corresponding Horizontal Pooling (HP) layers with a global pooling operation to achieve final feature invariance or approximate invariance.
• Figure 4: Visualizations of ReEL, RoEL, and SEL outputs (from up to bottom). RoEL or SEL shares the same heatmap across frames since their features undergo temporal pooling.
• Figure 5: Geometric invariance evaluations through applying varying probabilities of data augmentation (i.e., random reflection and rotation) during the test stage on CCPG.