Self Model for Embodied Intelligence: Modeling Full-Body Human Musculoskeletal System and Locomotion Control with Hierarchical Low-Dimensional Representation

TL;DR

This work addresses the challenge of modeling and controlling a full-body human musculoskeletal system with over $700$ muscles by introducing MS-Human-700, a 90-segment, 206-joint framework that supports neuromuscular-driven dynamics in physics engines. It advances control through Two-Stage Hierarchical Training (TSHT), combining a low-dimensional latent representation and physiological muscle synergy within a Soft Actor-Critic framework to produce natural, robust locomotion, including walking, exoskeleton interaction, and prosthetic walking. A reference generator from motion capture guides imitation via rewards, and two-stage training learns policies in a latent space that are decoded back to full muscle activations. The model and algorithm are released to the research community, enabling deeper study of human motor control and informing the design of interactive robots for embodied intelligence and improved human–robot interaction.

Abstract

Modeling and control of the human musculoskeletal system is important for understanding human motor functions, developing embodied intelligence, and optimizing human-robot interaction systems. However, current human musculoskeletal models are restricted to a limited range of body parts and often with a reduced number of muscles. There is also a lack of algorithms capable of controlling over 600 muscles to generate reasonable human movements. To fill this gap, we build a musculoskeletal model (MS-Human-700) with 90 body segments, 206 joints, and 700 muscle-tendon units, allowing simulation of full-body dynamics and interaction with various devices. We develop a new algorithm using low-dimensional representation and hierarchical deep reinforcement learning to achieve state-of-the-art full-body control. We validate the effectiveness of our model and algorithm in simulations with real human locomotion data. The musculoskeletal model, along with its control algorithm, will be made available to the research community to promote a deeper understanding of human motion control and better design of interactive robots. Project page: https://lnsgroup.cc/research/MS-Human-700

Figures (6)

• Figure 1: (a) MS-Human-700: full-body musculoskeletal model. Blue arrows represent joint axes, red lines represent muscle-tendon units (right half). (b) The model interacting with an exoskeleton in simulation. (c) The model with a prosthetic leg and a crutch.
• Figure 2: Learning to walk with MS-Human-700 via Two-Stage Hierarchical Training (TSHT).
• Figure 3: (a) The anatomy of Psoas Major muscle from the medical atlas, image courtesy of https://3d4medical.com/. (b) Its multi-muscle-tendon units and the wrapping geometry constructed in our model.
• Figure 4: Two-Stage Hierarchical Training (TSHT). In the collection stage, the agent determines muscle cluster actions. Muscle Synergy function then maps these muscle cluster actions into full-body muscle activation, leveraging prior physiological knowledge. Meanwhile, the reference generator generates the desired joint states based on motion capture data and computes corresponding rewards. Once a sufficient number of sub-optimal trajectories are collected, low-dimensional representations are extracted by encoder-decoder architecture and utilized for the training stage.
• Figure 5: (a)-(c) Our algorithm outperforms baseline algorithms on different tasks. (d) Learnt locomotion with a prosthetic leg and a crutch (top) and an exoskeleton (bottom).