Muscles in Time: Learning to Understand Human Motion by Simulating Muscle Activations

TL;DR

This work establishes Muscles in Time (MinT), a large-scale synthetic muscle activation dataset, and demonstrates the utility of this dataset by presenting results on neural network-based muscle activation estimation from human pose sequences with two different sequence-to-sequence architectures.

Abstract

Exploring the intricate dynamics between muscular and skeletal structures is pivotal for understanding human motion. This domain presents substantial challenges, primarily attributed to the intensive resources required for acquiring ground truth muscle activation data, resulting in a scarcity of datasets. In this work, we address this issue by establishing Muscles in Time (MinT), a large-scale synthetic muscle activation dataset. For the creation of MinT, we enriched existing motion capture datasets by incorporating muscle activation simulations derived from biomechanical human body models using the OpenSim platform, a common approach in biomechanics and human motion research. Starting from simple pose sequences, our pipeline enables us to extract detailed information about the timing of muscle activations within the human musculoskeletal system. Muscles in Time contains over nine hours of simulation data covering 227 subjects and 402 simulated muscle strands. We demonstrate the utility of this dataset by presenting results on neural network-based muscle activation estimation from human pose sequences with two different sequence-to-sequence architectures. Data and code are provided under https://simplexsigil.github.io/mint.

Figures (17)

• Figure 1: Simulation pipeline of the Muscles in Time dataset. The SMPL representation is extracted from videos, then, the SMPL represented motions are mapped to bio-mechanically validated human body models to simulate fine-grained muscle activation, connecting computer vision with biomechanical research. Bottom right: two activation sequences for exemplary muscles. Images from mandery2015kitdelp_opensim_2007
• Figure 2: The AMASS body model with specific indices mapped onto the OpenSim lower body model by Lai et al. lai2017antagonist (middle) and model of the thoracolumbar region by Bruno et al. bruno_development_2015 (right). Best viewed by zooming in.
• Figure 3: Approximated weight and height distribution of the analysed subjects in the MinT dataset.
• Figure 4: Prevalence of different motions in the MinT dataset.
• Figure 5: Left: Clustering of multiple activities within the BMLmovi dataset by muscle activation features. Right: Column-wise color coded histograms of areas under muscle activation curves for 402 muscle strains, sorted by histogram medians. Log-normalized color map, best displayed in color.