Explicit Time-Frequency Dynamics for Skeleton-Based Gait Recognition

Abstract

Skeleton-based gait recognizers excel at modeling spatial configurations but often underuse explicit motion dynamics that are crucial under appearance changes. We introduce a plug-and-play Wavelet Feature Stream that augments any skeleton backbone with time-frequency dynamics of joint velocities. Concretely, per-joint velocity sequences are transformed by the continuous wavelet transform (CWT) into multi-scale scalograms, from which a lightweight multi-scale CNN learns discriminative dynamic cues. The resulting descriptor is fused with the backbone representation for classification, requiring no changes to the backbone architecture or additional supervision. Across CASIA-B, the proposed stream delivers consistent gains on strong skeleton backbones (e.g., GaitMixer, GaitFormer, GaitGraph) and establishes a new skeleton-based state of the art when attached to GaitMixer. The improvements are especially pronounced under covariate shifts such as carrying bags (BG) and wearing coats (CL), highlighting the complementarity of explicit time-frequency modeling and standard spatio-temporal encoders.

Figures (2)

• Figure 1: Overview of the proposed plug-and-play Wavelet Feature Stream attached to a skeleton backbone. Per-joint velocities $\rightarrow$ CWT scalograms $\rightarrow$ multi-scale CNN $\rightarrow$ fusion with backbone features.
• Figure 2: Wavelet Feature Stream. Given per-joint velocities from skeleton sequences, we apply CWT to obtain time--frequency maps (scalograms) and learn multi-scale dynamics via a lightweight CNN. The resulting descriptor is fused with a backbone feature for classification.