Interpretable and Granular Video-Based Quantification of Motor Characteristics from the Finger Tapping Test in Parkinson Disease

TL;DR

A granular computer vision-based method for quantifying PD motor characteristics from video recordings that achieves a higher accuracy in MDS-UPDRS score prediction, while still providing an interpretable quantification of individual finger-tapping motor characteristics.

Abstract

Accurately quantifying motor characteristics in Parkinson disease (PD) is crucial for monitoring disease progression and optimizing treatment strategies. The finger-tapping test is a standard motor assessment. Clinicians visually evaluate a patient's tapping performance and assign an overall severity score based on tapping amplitude, speed, and irregularity. However, this subjective evaluation is prone to inter- and intra-rater variability, and does not offer insights into individual motor characteristics captured during this test. This paper introduces a granular computer vision-based method for quantifying PD motor characteristics from video recordings. Four sets of clinically relevant features are proposed to characterize hypokinesia, bradykinesia, sequence effect, and hesitation-halts. We evaluate our approach on video recordings and clinical evaluations of 74 PD patients from the Personalized Parkinson Project. Principal component analysis with varimax rotation shows that the video-based features corresponded to the four deficits. Additionally, video-based analysis has allowed us to identify further granular distinctions within sequence effect and hesitation-halts deficits. In the following, we have used these features to train machine learning classifiers to estimate the Movement Disorder Society Unified Parkinson Disease Rating Scale (MDS-UPDRS) finger-tapping score. Compared to state-of-the-art approaches, our method achieves a higher accuracy in MDS-UPDRS score prediction, while still providing an interpretable quantification of individual finger-tapping motor characteristics. In summary, the proposed framework provides a practical solution for the objective assessment of PD motor characteristics, that can potentially be applied in both clinical and remote settings. Future work is needed to assess its responsiveness to symptomatic treatment and disease progression.

Figures (11)

• Figure 1: Block diagram describing the analysis pipline of the proposed method. The system detects key points—including the index finger, thumb, and wrist—in each video frame and tracks them throughout the video sequence. Next, a time-series signal is generated, representing the distance between the thumb and index finger, which reflects the patient’s ability to fully open their fingers. This signal is scaled by the palm length to account for variations in the patient’s distance from the camera. Features are extracted from the signal to assess PD severity across four main domains: hypokinesia, bradykinesia, sequence effect, and hesitation/halts. Per video, the MDS-UPDRS score values are then estimated by machine learning classifiers, which had been trained on these features. The classification performances are estimated using a leave-one-patient-out cross-validation scheme. Finally, the trained classifiers are introspected.
• Figure 2: Amplitude, speed, and tapping interval for two patients with MDS-UPDRS scores of 0 and 4. The distance signal for the patient with a MDS-UPDRS score of 0 demonstrates high-amplitude tapping, whereas the patient with a MDS-UPDRS score of 4 shows lower amplitudes. Similarly, the speed signal for the patient with a MDS-UPDRS score of 0 exhibits stable, high-speed tapping that highlights the ability of the patient to maintain a steady tapping throughout the task. In contrast, the speed signal for the patient with a MDS-UPDRS score of 4 reveals a significantly reduced tapping speed, which reduces over time. The tapping interval signals also reveal distinct patterns. The patient with a MDS-UPDRS score of 0 exhibits an average duration of 0.28 seconds. In contrast, the patient with a MDS-UPDRS score of 4 shows a prolonged average tapping interval of 0.56 seconds.
• Figure 3: Distribution of clinically interpretable features across MDS-UPDRS finger-tapping scores. Each subplot shows the distribution of a specific feature across MDS-UPDRS scores (0–4), highlighting its relation to motor impairment severity. (a) Hypokinesia: average amplitude, measuring hand opening ability. (b) Bradykinesia: Average tapping interval. (c, d) Combined hypo- & bradykinesia: CAS and CMS. (e-g) Sequence effect: amplitude decrement, tapping interval increment, speed decrement. (h–l) Hesitation-halts: COV in tapping interval, amplitude, CMS, CAS, and number of interruptions. Each feature reflects a clinically relevant aspect of motor impairment, with distributions aligning with increasing disease severity
• Figure 4: PCA with varimax rotation for feature groupings and potential reclassification of motor deficits in PD. Varimax rotation is applied to enhance interpretability by maximizing the separation between feature across components. The heatmap displays the loading values of each feature across the rotated principal components. Warmer colors (red) indicate strong positive loadings, and cooler colors (blue) represent strong negative loadings. Features with high loadings on the same component suggest shared variance and likely represent related motor characteristics.
• Figure 5: Scree plot of principal components. The blue bars represent the individual explained variance ratio for each principal component, indicating how much variance in the data is captured by that component. The red line shows the cumulative explained variance, which represents the total amount of variance accounted for when combining the first n components. This plot demonstrates that the first 9 components together explain approximately 98% of the total variance.