A Personalized Data-Driven Generative Model of Human Repetitive Motion

TL;DR

This work proposes a fully data-driven approach, based on long short-term memory neural networks, to generate original motion that captures the unique features of specific individuals, and validate the architecture using real human data from participants performing spontaneous oscillatory motion.

Abstract

The deployment of autonomous virtual avatars (in extended reality) and robots in human group activities -- such as rehabilitation therapy, sports, and manufacturing -- is expected to increase as these technologies become more pervasive. Designing cognitive architectures and control strategies to drive these agents requires realistic models of human motion. Furthermore, recent research has shown that each person exhibits a unique velocity signature, highlighting how individual motor behaviors are both rich in variability and internally consistent. However, existing models only provide simplified descriptions of human motor behavior, hindering the development of effective cognitive architectures. In this work, we first show that motion amplitude provides a valid and complementary characterization of individual motor signatures. Then, we propose a fully data-driven approach, based on long short-term memory neural networks, to generate original motion that captures the unique features of specific individuals. We validate the architecture using real human data from participants performing spontaneous oscillatory motion. Extensive analyses show that state-of-the-art Kuramoto-like models fail to replicate individual motor signatures, whereas our model accurately reproduces the velocity distribution and amplitude envelopes of the individual it was trained on, while remaining distinct from others.

Figures (6)

• Figure 1: Schematic representation of the experimental setup considered.dataset_article Five people had to move their preferred index finger continuously from left to right over a motion sensor without any interaction with other participants. The motion sensor works through infrared technology and has a perception area depicted in light-blue. Each individual performed 7 trials, each lasting 30 s.
• Figure 2: Representation of the dataset in the similarity planes of the velocity profiles (left) and amplitudes (right). In both planes, each point represents the movement of an individual (P $i$), indicated by color, recorded over an entire trial. Coordinates in the velocity-profiles plane reproduce earth mover's distances between any velocity profile in the dataset; coordinates in the amplitudes plane are mean positive and negative amplitudes of the position signals. The points of each individual are enclosed by a covariance ellipse of the corresponding color: the smaller the ellipse, the more consistent the individual. Overlap and closeness between ellipses highlight similarities between motion features of two individuals.
• Figure 3: Comparison of position timeseries (a) and velocity profiles (b) between a human, the generative model and a linear oscillator. In panel (a), the position timeseries of trial $2$ executed by individual $2$ is compared with two other signals: the position timeseries produced by a linear oscillator whose frequency is sampled from a Gaussian distribution fitted on the individual's data; the position timeseries produced by the generative model, seeded with the first 4 s of the human signal. Panel (b) depicts the velocity profiles of the corresponding signals. The velocity profiles are obtained numerically by binning velocity values and counting occurrences. In the oscillator's panel, the theoretical probability density function of a linear oscillator's velocity is depicted in magenta, showing how the shape of the binned velocity profile matches the theoretical results.
• Figure 4: Representation of generated signals in the similarity planes of velocity profiles and amplitudes. Each panel contains the representation of signals from individuals (P $i$), the generative models (GM $i$) and linear oscillators (O $i$). For the sake of clarity, upper panels contain signals from individuals 1 and 5 while bottom panels contain signals from individuals 2,3, and 4. Note that, in the amplitude-based plane, all points associated to a single oscillator coincide, as their motion amplitude is fixed between trials.
• Figure 5: Overlap $\Omega_{ij}$ and center distance $\delta_{ij}$ between ellipses in both the similarity planes of velocity profiles and amplitudes. The values on the two main diagonals of the heatmaps are highlighted in white and depicted in the corresponding bar plots; these values show the ability of each model (generative model, GM $i$, or linear oscillator, O $i$) to reproduce the individual motor signature of the target individual (P $i$)---high $\Omega_{ij}$ and low $\delta_{ij}$ signal good modeling performance. Remarkably, linear oscillators' ellipses never overlap with the corresponding individuals' (yielding $\Omega_{ij} = 0$). On the other hand, the generative models' ellipses tend to overlap and stay close to those of the modeled individuals (yielding comparatively high $\Omega_{ij}$ and/or low $\delta_{ij}$).