Evolutionary Morphology Towards Overconstrained Locomotion via Large-Scale, Multi-Terrain Deep Reinforcement Learning

TL;DR

This work investigates overconstrained locomotion as a morphologically informed, reconfigurable paradigm for energy-efficient locomotion. It introduces a 3D-printable Bennett-based overconstrained limb that can morph into planar or spherical 4R configurations and can be upgraded to wheel-legged forms, all actuated coaxially. A large-scale, multi-terrain deep reinforcement learning framework demonstrates that overconstrained limbs achieve superior energy efficiency and higher top speeds compared to planar and spherical counterparts across floors, slopes, and stairs, with quantified gains such as up to $22\%$ reductions in mechanical energy and a top forward speed of $0.85\ \mathrm{m/s}$ on flat terrain. The study further extends to wheel-legged locomotion and discusses data-driven mechanism intelligence as a pathway to realizing evolutionary morphology in reconfigurable robots, while acknowledging sim-to-real challenges and the need for hardware validation for practical deployment.

Abstract

While the animals' Fin-to-Limb evolution has been well-researched in biology, such morphological transformation remains under-adopted in the modern design of advanced robotic limbs. This paper investigates a novel class of overconstrained locomotion from a design and learning perspective inspired by evolutionary morphology, aiming to integrate the concept of `intelligent design under constraints' - hereafter referred to as constraint-driven design intelligence - in developing modern robotic limbs with superior energy efficiency. We propose a 3D-printable design of robotic limbs parametrically reconfigurable as a classical planar 4-bar linkage, an overconstrained Bennett linkage, and a spherical 4-bar linkage. These limbs adopt a co-axial actuation, identical to the modern legged robot platforms, with the added capability of upgrading into a wheel-legged system. Then, we implemented a large-scale, multi-terrain deep reinforcement learning framework to train these reconfigurable limbs for a comparative analysis of overconstrained locomotion in energy efficiency. Results show that the overconstrained limbs exhibit more efficient locomotion than planar limbs during forward and sideways walking over different terrains, including floors, slopes, and stairs, with or without random noises, by saving at least 22% mechanical energy in completing the traverse task, with the spherical limbs being the least efficient. It also achieves the highest average speed of 0.85 meters per second on flat terrain, which is 20% faster than the planar limbs. This study paves the path for an exciting direction for future research in overconstrained robotics leveraging evolutionary morphology and reconfigurable mechanism intelligence when combined with state-of-the-art methods in deep reinforcement learning.

Figures (5)

• Figure 1: Parametrically reconfigurable additive design of overconstrained robotic limb. (A) 3D printable components and assembly of an overconstrained robotic limb; (B) Physical prototype of the fully-assembled quadruped in Bennett limbs; (C) Parametric reconfiguration as (i) a planar four-bar, (ii) an overconstrained 4-bar, and (iii) a spherical four-bar; (D) An enhanced design for amphibious locomotion reconfigurable as (i) a legged robot, (ii) a wheel-legged robot, and (iii) a simplified equivalent as an open-chain limb identical if using a belt or slender linkages.
• Figure 2: Simulation setup for massively parallel deep reinforcement learning. (A) Screenshot of the training scene for reinforcement learning in a large-scale, multi-terrain environment using (i) 2048 overconstrained-legged robots and (ii) 4096 overconstrained wheel-legged robots. (B) Closed-up view of the training scene with (i) overconstrained-legged robots and (ii) overconstrained wheel-legged robots; (C) Schematic of the multi-terrain training process; (D) Flow chart for training overconstrained locomotion.
• Figure 3: Large-scale, multi-terrain reinforcement learning of overconstrained locomotion. Besides walking on the simple flat floor, here are the screenshots of the overconstrained quadruped walking on (A) simple slopes, (B) slopes with random noises, (C) stairs, and (D) flat floors with random noises. (E) reports the reinforcement learning rewards of the quadruped with Bennett, planar, and spherical limbs over 6K training steps.
• Figure 4: Performance benchmark on terrain traverse task. (A) 30 robots with Bennett limbs are completing the traverse task by walking sideways; (B) Terrain distribution of the traverse task used to benchmark the locomotion policies; (C) Trajectories of quadrupeds with Bennett limbs walking forward (i) and sideways(ii); (D) Trajectories of quadrupeds with planar or spherical limbs walking forward (i, ii) and sideways(iii, iv); (E) Average horizontal linear velocity of quadrupeds concerning the distance walked along the traverse direction; (F) Average Cost-of-Transport concerning the distance walked along the traverse direction.
• Figure 5: Learning wheel-legged overconstrained locomotion. (A) Testing Policy on Complex Terrain: (i) Evaluating performance on stairs. (ii) Assessing obstacle navigation capabilities through random obstacles. (iii) Evaluating adaptability on uneven and rugged terrain. (B) Training Data Analysis Throughout Training Process: (i) Mean reward progression over time steps. (ii) The precision of tracking linear velocity commands.