Watch Where You Move: Region-aware Dynamic Aggregation and Excitation for Gait Recognition

TL;DR

This work addresses the sensitivity of gait recognition to covariates by introducing GaitRDAE, a region-aware framework that jointly learns adaptive temporal scales and targeted attention for motion regions. The Region-aware Dynamic Aggregation (RDA) searches per-region temporal receptive fields, while Region-aware Dynamic Excitation (RDE) enhances motion-region learning through Spatial-wise (SME) and Channel-wise (CME) excitations. Across GREW, Gait3D, CASIA-B, and OU-MVLP, GaitRDAE achieves state-of-the-art results, with ablations confirming that RDA and RDE provide complementary gains and robust improvements under diverse conditions. The work demonstrates practical impact by improving gait recognition robustness to clothing, viewpoint, and background covariates, enabling more reliable real-world deployment. Mathematical notations such as the learnable temporal offset $\Delta t$ and the combined loss $L_{combined}$ are used to formalize the adaptive fusion and optimization strategy.

Abstract

Deep learning-based gait recognition has achieved great success in various applications. The key to accurate gait recognition lies in considering the unique and diverse behavior patterns in different motion regions, especially when covariates affect visual appearance. However, existing methods typically use predefined regions for temporal modeling, with fixed or equivalent temporal scales assigned to different types of regions, which makes it difficult to model motion regions that change dynamically over time and adapt to their specific patterns. To tackle this problem, we introduce a Region-aware Dynamic Aggregation and Excitation framework (GaitRDAE) that automatically searches for motion regions, assigns adaptive temporal scales and applies corresponding attention. Specifically, the framework includes two core modules: the Region-aware Dynamic Aggregation (RDA) module, which dynamically searches the optimal temporal receptive field for each region, and the Region-aware Dynamic Excitation (RDE) module, which emphasizes the learning of motion regions containing more stable behavior patterns while suppressing attention to static regions that are more susceptible to covariates. Experimental results show that GaitRDAE achieves state-of-the-art performance on several benchmark datasets.

Figures (6)

• Figure 1: (a) An intuitive diagram of our methods. RDA is used to dynamically search the temporal receptive field of each region. RDE is utilized to adaptively enhance the attention of the motion region. (b) Attention heatmaps from the existing method and our method are illustrated. The gradient bar to the right showcases how attention is variably allocated among diverse color segments.
• Figure 2: An overview of the GaitRDAE architecture is presented. Our system processes a sequence of T gait frames as its input. The term "Conv" represents the convolutional layers within the framework. Feature fusion in the model employs element-wise addition, denoted by “+”, and height-level concatenation, represented by “C”. The loss functions used in this model are the triplet loss ($L_{tri}$) and the cross-entropy loss ($L_{ce}$).
• Figure 3: An intuitive illustration of Spatial-wise Motion Excitation (SME) and Channel-wise Motion Excitation (CME) .
• Figure 4: Visualization of the learnable temporal offset value.
• Figure 5: Feature distribution visualization is achieved using t-SNEtsne: (a) Baseline's feature distribution. (b) GaitRDAE's feature distribution.