Benchmarking Skeleton-based Motion Encoder Models for Clinical Applications: Estimating Parkinson's Disease Severity in Walking Sequences

TL;DR

This study benchmarks six general motion encoder models, pre-trained on large-scale healthy-m population data, for PD gait severity estimation using the pdDataset with UPDRS-gait annotations. It compares encoder-based predictions to a traditional gait-feature baseline (TransformingGait features with Random Forest) under Leave-One-Subject-Out cross-validation, and explores fine-tuning the best encoder (PoseFormerV2) on PD data. Results show the feature-based approach often outperforms the encoders in raw accuracy metrics, though PoseFormerV2-Finetuned approaches encoder performance and demonstrates sensitivity to medication state changes (ON vs OFF). The work establishes the first public benchmark for skeleton-based motion encoders in a clinical PD setting and provides a leaderboard and code to facilitate future developments and clinical translation.

Abstract

This study investigates the application of general human motion encoders trained on large-scale human motion datasets for analyzing gait patterns in PD patients. Although these models have learned a wealth of human biomechanical knowledge, their effectiveness in analyzing pathological movements, such as parkinsonian gait, has yet to be fully validated. We propose a comparative framework and evaluate six pre-trained state-of-the-art human motion encoder models on their ability to predict the Movement Disorder Society - Unified Parkinson's Disease Rating Scale (MDS-UPDRS-III) gait scores from motion capture data. We compare these against a traditional gait feature-based predictive model in a recently released large public PD dataset, including PD patients on and off medication. The feature-based model currently shows higher weighted average accuracy, precision, recall, and F1-score. Motion encoder models with closely comparable results demonstrate promise for scalability and efficiency in clinical settings. This potential is underscored by the enhanced performance of the encoder model upon fine-tuning on PD training set. Four of the six human motion models examined provided prediction scores that were significantly different between on- and off-medication states. This finding reveals the sensitivity of motion encoder models to nuanced clinical changes. It also underscores the necessity for continued customization of these models to better capture disease-specific features, thereby reducing the reliance on labor-intensive feature engineering. Lastly, we establish a benchmark for the analysis of skeleton-based motion encoder models in clinical settings. To the best of our knowledge, this is the first study to provide a benchmark that enables state-of-the-art models to be tested and compete in a clinical context. Codes and benchmark leaderboard are available at code.

Figures (2)

• Figure 1: Overview of the framework for UPDRS-gait score estimation. (top) employs motion encoder models to predict UPDRS-gait scores directly from gait motion, while (bottom) uses a feature-based approach for score estimation from gait features estimated from motion data. (right) encapsulates the full pipeline from data preprocessing to model evaluation, employing LOSOCV and majority voting to validate and compare the effectiveness of both approaches against clinical UPDRS-gait scores.
• Figure 2: Normalized confusion matrices for UPDRS-gait score predictions from PoseFormerV2-Finetuned (top row) and feature-based models (bottom row) across overall (left), OFF (middle), and ON (right) medication states.