Investigating Premature Convergence in Co-optimization of Morphology and Control in Evolved Virtual Soft Robots

TL;DR

This study probes why co-optimizing morphology and control in evolved virtual soft robots often stalls due to premature convergence of body plans. By contrasting learnable neural controllers with proprioceptive inputs against fixed, non-sensing controllers across multiple morphologies and environments, the authors reveal high-performing morphologies that remain undiscovered during co-optimization. A key contribution is a body-centric interpretation of the fitness landscape, illustrating a first-mover effect where early-found body plans dominate the search, and suggesting pathways such as resource augmentation and morphology-robust controllers to mitigate this. The findings underscore the importance of rethinking search strategies in brain–body co-optimization and offer a framework to guide future improvements in efficient design of soft-robotic agents.

Abstract

Evolving virtual creatures is a field with a rich history and recently it has been getting more attention, especially in the soft robotics domain. The compliance of soft materials endows soft robots with complex behavior, but it also makes their design process unintuitive and in need of automated design. Despite the great interest, evolved virtual soft robots lack the complexity, and co-optimization of morphology and control remains a challenging problem. Prior work identifies and investigates a major issue with the co-optimization process -- fragile co-adaptation of brain and body resulting in premature convergence of morphology. In this work, we expand the investigation of this phenomenon by comparing learnable controllers with proprioceptive observations and fixed controllers without any observations, whereas in the latter case, we only have the optimization of the morphology. Our experiments in two morphology spaces and two environments that vary in complexity show, concrete examples of the existence of high-performing regions in the morphology space that are not able to be discovered during the co-optimization of the morphology and control, yet exist and are easily findable when optimizing morphologies alone. Thus this work clearly demonstrates and characterizes the challenges of optimizing morphology during co-optimization. Based on these results, we propose a new body-centric framework to think about the co-optimization problem which helps us understand the issue from a search perspective. We hope the insights we share with this work attract more attention to the problem and help us to enable efficient brain-body co-optimization.

Figures (5)

• Figure 1: Fitness over time plots (left) and distributions of the best solutions (right) for the co-optimization experiments. The simplest controller, the fixed controller, significantly outperforms the learnable controller in all experimental settings. Moreover, populations with fixed controllers converge faster. Left: Solid lines show the best fitness found at each generation, averaged across 10 runs. Shaded regions show the $95\%$ confidence intervals. Right: Each data point is plotted, as well as the mean values which are marked with dark red. Horizontal lines indicate statistically different results. ***: $P<0.001$, **: $P<0.005$, *: $P<0.05$
• Figure 2: t-SNE plots of the run champions' body plans and average intra-cluster distances in the embedding space. The populations with the fixed controller converge better to a body plan across different runs and morphology spaces.
• Figure 3: Morphologies of the run champions under each experimental setting. The left half of the figure shows the run champions with learnable controllers, and the right half shows the ones with fixed controllers. We see diverse body plans evolve with the learnable controller, especially in (a). On the other hand, we see the same form of the mostly active bottom with two vertical apparatus at the front and back, resembling a head and a tail, evolved with the fixed controller in (e). For the BridgeWalker-v0 environment, individuals with learnable controllers show similar body plan features in (b) and (d), e.g. upright posture, r or T shape, individuals with fixed controller usually consists of a thin, horizontal body with active materials and forward apparatus resembling a leg in (f-h).
• Figure 4: Timelapse images of three run champions' behavior from each of the two tasks of the $(5,5)$ morphology space, co-optimized brain and body (a) and morphology-only optimization with fixed control (b). Learnable controllers demonstrate more diversity, especially in the Walker-v0 environment. Most of the individuals with learnable controllers (a) find a bipedal (rows 1, 2, 4-6) or monopedal gait (row 3). Fixed controllers (b) show one common behavior on flat ground (rows 1-3), consisting of an active bottom with two vertical apparatus at the front and back, resembling a head and a tail. When the muscle at the bottom of the robot contracts, vertical apparatuses help the bottom part form an arch, creating front and back legs to locomote. In the bridge environment (rows 4-6) often includes a thin, horizontal body with active materials, a forward apparatus resembling a leg, and some upper body (presumably used for balancing). Individuals with this body plan throw themselves forward by actuating their muscles in phase and pulling themselves forward with their forward apparatus, achieving bipedal locomotion.
• Figure 5: Comparison of performances on the body plans of learnable and fixed run champions with learnable and fixed controllers. The first and last columns in each plot show the results of the main co-optimization experiment, and the second and third columns show the results when the learnable controller is optimized from scratch on body plans of learnable run champions and fixed run champions, respectively. In 3 out of 4 experimental settings, the body plans found by the fixed controller during the co-optimization experiments outperform the body plans found by the learnable controller when the learnable controller is optimized to control them. It demonstrates the failure of search over the morphology space during co-optimization with learnable controllers since they failed to discover high-performing body plans of fixed champions.