DynSyn: Dynamical Synergistic Representation for Efficient Learning and Control in Overactuated Embodied Systems

TL;DR

DynSyn tackles efficient control of high-dimensional, overactuated neuromusculoskeletal systems by deriving dynamical synergistic representations of actuators from random perturbation trajectories and embedding them into a state-dependent reinforcement learning framework. Actuator groups are formed by clustering muscle-length trajectory correlations via K-Medoids, yielding unified group actions with per-muscle correction weights, all integrated within a SAC-based policy. Across six MuJoCo benchmarks with up to 700 muscles, DynSyn achieves state-of-the-art sample efficiency and robust generalization to terrain and task variations, while producing interpretable synergies aligned with biomechanical structure. This approach offers a principled, scalable path for efficient motor control in both artificial and biological embodied systems.

Abstract

Learning an effective policy to control high-dimensional, overactuated systems is a significant challenge for deep reinforcement learning algorithms. Such control scenarios are often observed in the neural control of vertebrate musculoskeletal systems. The study of these control mechanisms will provide insights into the control of high-dimensional, overactuated systems. The coordination of actuators, known as muscle synergies in neuromechanics, is considered a presumptive mechanism that simplifies the generation of motor commands. The dynamical structure of a system is the basis of its function, allowing us to derive a synergistic representation of actuators. Motivated by this theory, we propose the Dynamical Synergistic Representation (DynSyn) algorithm. DynSyn aims to generate synergistic representations from dynamical structures and perform task-specific, state-dependent adaptation to the representations to improve motor control. We demonstrate DynSyn's efficiency across various tasks involving different musculoskeletal models, achieving state-of-the-art sample efficiency and robustness compared to baseline algorithms. DynSyn generates interpretable synergistic representations that capture the essential features of dynamical structures and demonstrates generalizability across diverse motor tasks.

Figures (14)

• Figure 1: Motor behaviors of overactuated musculoskeletal systems acquired by DynSyn. (a) Gait of MS-Human-700 model. (b) Manipulation of MS-Human-700 Arm model. (c) Locomotion of Ostrich model. See our anonymous project website at https://sites.google.com/view/dynsyn.
• Figure 2: Musculoskeletal model. (a) Full body model MS-Human-700, red lines represent muscle-tendon units. (b) Arm model with wrist and finger joints for dexterous manipulation. The top illustrates the the skeleton structure and the bottom illustrates muscles.
• Figure 3: Motivation of DynSyn. The brown link represents a robot arm (or bone), while the blue and green lines represent the cable actuators (or muscles). By randomly controlling the joint velocity, the lengths of the four actuators are demonstrated on the right. Actuators with similar functions are categorized into the same group due to similar structures, based on the correlation of length changes.
• Figure 4: [DynSyn] Dynamical Synergistic Representation
• Figure 5: Overview of DynSyn. The algorithm generates a unified action $a_G$ for each group of actuators, along with state-dependent correction weights $w$ for each actuator on top of the unified action $a_G$. Separate MLPs are used to generate the parameters of two Gaussian distributions. We then sample from the Gaussian distributions and pass them through a squashing function Tanh to obtain $a_G$ and $w$.