Integrating Sample Inheritance into Bayesian Optimization for Evolutionary Robotics

TL;DR

The paper tackles the problem of co-optimizing morphology and control in evolutionary robotics under a limited controller-learning budget. It introduces two Lamarckian sample-inheritance schemes within a Bayesian optimization framework to transfer experience across generations, with reevaluation of selected samples providing the strongest gains. Across four environments and two controller types, reevaluate consistently outperforms other approaches, while inherit-samples also offers benefits over no inheritance, particularly for morphologies similar to their parents. The findings demonstrate that inheritance can accelerate learning and improve performance without large data budgets, highlighting its potential to enable more capable robot design in resource-constrained settings.

Abstract

In evolutionary robotics, robot morphologies are designed automatically using evolutionary algorithms. This creates a body-brain optimization problem, where both morphology and control must be optimized together. A common approach is to include controller optimization for each morphology, but starting from scratch for every new body may require a high controller learning budget. We address this by using Bayesian optimization for controller optimization, exploiting its sample efficiency and strong exploration capabilities, and using sample inheritance as a form of Lamarckian inheritance. Under a deliberately low controller learning budget for each morphology, we investigate two types of sample inheritance: (1) transferring all the parent's samples to the offspring to be used as prior without evaluating them, and (2) reevaluating the parent's best samples on the offspring. Both are compared to a baseline without inheritance. Our results show that reevaluation performs best, with prior-based inheritance also outperforming no inheritance. Analysis reveals that while the learning budget is too low for a single morphology, generational inheritance compensates for this by accumulating learned adaptations across generations. Furthermore, inheritance mainly benefits offspring morphologies that are similar to their parents. Finally, we demonstrate the critical role of the environment, with more challenging environments resulting in more stable walking gaits. Our findings highlight that inheritance mechanisms can boost performance in evolutionary robotics without needing large learning budgets, offering an efficient path toward more capable robot design.

Figures (11)

• Figure 1: Overview of an evolutionary loop for morphology optimization with Bayesian optimization-driven controller learning, and controller inheritance methods.
• Figure 2: Overview of Bayesian optimization and the inheritance methods. Bayesian optimization iteratively samples the control parameters with the highest acquisition value, which considers the surrogate function's mean and uncertainty. Inheritance is done using two methods. With inherit samples, all samples with their measured performance are transferred without evaluating them, this is the same as transferring the parent's posterior to be the offspring's prior. With reevaluate, the best-performing samples from the parent are reevaluated by the offspring, leaving the rest of the learning budget for the Bayesian optimization process.
• Figure 3: An example robot with its CPG network. The white modules are the actuators. A blue solid line represents a Manhattan distance of 1, and a red dotted line represents a Manhattan distance of 2.
• Figure 4: There are three types of modules, and every robot always has one head module. The head module has four attachment points, the block module has six (it includes up and down) and the joint module has two. Symmetry is ensured by making the left and right parts of the central axis identical. Every joint module has its own sine wave parameters, but the symmetrical parts share parameters.
• Figure 5: The four environments used in this work.