MoME: Estimating Psychological Traits from Gait with Multi-Stage Mixture of Movement Experts

TL;DR

The paper addresses the challenge of inferring psychological traits from gait, proposing MoME, a hierarchical four-stage mixture of movement experts with stage-wise and task-specific gates to route movement features for multi-task prediction. The method leverages 2D pose sequences and auxiliary tasks to diversify learned representations, achieving state-of-the-art run-level F1 of 37.47% and subject-level F1 of 44.6% on PsyMo across 17 traits. Results show that auxiliary tasks such as re-identification, gender, and BMI improve psychological trait estimation and reveal interpretable patterns of feature sharing across tasks. Overall, the work demonstrates the viability of multi-task gait-focused learning for psychological inference and provides a framework for interpretable movement-informed psychometrics.

Abstract

Gait encodes rich biometric and behavioural information, yet leveraging the manner of walking to infer psychological traits remains a challenging and underexplored problem. We introduce a hierarchical Multi-Stage Mixture of Movement Experts (MoME) architecture for multi-task prediction of psychological attributes from gait sequences represented as 2D poses. MoME processes the walking cycle in four stages of movement complexity, employing lightweight expert models to extract spatio-temporal features and task-specific gating modules to adaptively weight experts across traits and stages. Evaluated on the PsyMo benchmark covering 17 psychological traits, our method outperforms state-of-the-art gait analysis models, achieving a 37.47% weighted F1 score at the run level and 44.6% at the subject level. Our experiments show that integrating auxiliary tasks such as identity recognition, gender prediction, and BMI estimation further improves psychological trait estimation. Our findings demonstrate the viability of multi-task gait-based learning for psychological trait estimation and provide a foundation for future research on movement-informed psychological inference.

Figures (4)

• Figure 1: High-level overview of our proposed method consisting of a multi-stage Mixture of Movement Experts framework. At each stage, representing a different level of movement complexity, every task has an assigned gating module which computes the optimal weight of the expert outputs for making the prediction.
• Figure 2: Visualization of single stage module. Movement features obtained from a previous stage are processed by multiple experts to obtain more complex features. Independent Task Gates weight each of the expert outputs for obtaining task-specific representations while the Main Gate obtains the weights of the movement features for the next stage.
• Figure 3: Visualization of the expert and gating module architectures. Following the hierarchical design of GaitPT catruna2024gaitpt our expert and gating modules are designed to extract relevant spatio-temporal features by utilizing an encoder with spatial attention and one with temporal attention.
• Figure 4: Expert activation heatmap across stages and tasks in our proposed MoME architecture. Each cell represents the post-softmax gating weight assigned to a specific expert for a given psychological attribute at a specific stage of movement complexity.