Database-Agnostic Gait Enrollment using SetTransformers

TL;DR

The paper addresses open-set gait enrollment, where a system must decide if a probe belongs to an enrolled identity or represents a new subject. It introduces a dataset-agnostic, model-agnostic enrollment framework based on a SetTransformer that attends to a probe embedding and a context of $K$ nearest gallery embeddings, decoupled from the main recognition pipeline. The method is evaluated on CASIA-B and PsyMo using skeleton-based representations from three backbones (GaitGraph, GaitFormer, GaitPT) and demonstrates robustness across gallery sizes, identity distributions, and embedding models, with larger $K$ and identity-aware neighbor coupling yielding improved performance. The study highlights practical potential for deploying open-set gait enrollment systems and plans for code and dataset scenario release to support replication and broader adoption.

Abstract

Gait recognition has emerged as a powerful tool for unobtrusive and long-range identity analysis, with growing relevance in surveillance and monitoring applications. Although recent advances in deep learning and large-scale datasets have enabled highly accurate recognition under closed-set conditions, real-world deployment demands open-set gait enrollment, which means determining whether a new gait sample corresponds to a known identity or represents a previously unseen individual. In this work, we introduce a transformer-based framework for open-set gait enrollment that is both dataset-agnostic and recognition-architecture-agnostic. Our method leverages a SetTransformer to make enrollment decisions based on the embedding of a probe sample and a context set drawn from the gallery, without requiring task-specific thresholds or retraining for new environments. By decoupling enrollment from the main recognition pipeline, our model is generalized across different datasets, gallery sizes, and identity distributions. We propose an evaluation protocol that uses existing datasets in different ratios of identities and walks per identity. We instantiate our method using skeleton-based gait representations and evaluate it on two benchmark datasets (CASIA-B and PsyMo), using embeddings from three state-of-the-art recognition models (GaitGraph, GaitFormer, and GaitPT). We show that our method is flexible, is able to accurately perform enrollment in different scenarios, and scales better with data compared to traditional approaches. We will make the code and dataset scenarios publicly available.

Figures (6)

• Figure 1: Overview of enrollment training and evaluation scenarios. Starting from an initial gallery of identity embeddings, probes are evaluated under database setups containing more identities but with fewer walks per identity and fewer identities containing more walks per identity. Each scenario is described with a certain id:walk ratio. Positive probes match existing gallery identities; negative probes represent new identities.
• Figure 2: Overall diagram of our model. A probe embedding attends to its $K$ nearest neighbor gait embeddings and corresponding identity‐average embeddings via a SetTransformer to predict enrollment status. The correspondence between identities is facilitated by learned positional embeddings.
• Figure 3: Performance across ID: Walk ratios (0.25, 0.5, 0.75) on CASIA-B and PsyMo for GaitPT, GaitGraph, and GaitFormer. On average, the more imbalanced the setup, the more performance drops.
• Figure 4: The effect of increasing the number of neighbors $K$ in the SetTransformer, on CASIA-B and PsyMo, for GaitPT, GaitGraph and GaitFormer, with embedding sizes 128, 256, 512. More neighbors present in the context results in better enrollment performance.
• Figure 5: Comparison of neighbor–identity coupling methods (Additive, Per-Instance, Per-Identity) in terms of MCC mcc1975 on CASIA-B and PsyMo using GaitPT, GaitGraph and GaitFormer generated embeddings.