Controller Distillation Reduces Fragile Brain-Body Co-Adaptation and Enables Migrations in MAP-Elites

TL;DR

The paper addresses fragile brain-body co-adaptation in brain-body co-optimization and shows that standard MAP-Elites struggles due to declining migrations, which undermine stepping-stone-based search. It introduces Pollination, a distillation-based extension that periodically transfers generalized controllers back into the archive to promote zero-shot transfer and inter-niche migrations. Empirical results demonstrate that Pollination increases migrations and improves quality-diversity metrics (total and maximal quality, reliability) compared to standard MAP-Elites, though it also reveals a trade-off where generalist controllers can be more fragile to brain mutations. The work provides insights into how quality-diversity methods can be adapted for embodied intelligence problems and suggests broader applicability and future refinements in balancing brain and body mutational effects.

Abstract

Brain-body co-optimization suffers from fragile co-adaptation where brains become over-specialized for particular bodies, hindering their ability to transfer well to others. Evolutionary algorithms tend to discard such low-performing solutions, eliminating promising morphologies. Previous work considered applying MAP-Elites, where niche descriptors are based on morphological features, to promote better search over morphology space. In this work, we show that this approach still suffers from fragile co-adaptation: where a core mechanism of MAP-Elites, creating stepping stones through solutions that migrate from one niche to another, is disrupted. We suggest that this disruption occurs because the body mutations that move an offspring to a new morphological niche break the robots' fragile brain-body co-adaptation and thus significantly decrease the performance of those potential solutions -- reducing their likelihood of outcompeting an existing elite in that new niche. We utilize a technique, we call Pollination, that periodically replaces the controllers of certain solutions with a distilled controller with better generalization across morphologies to reduce fragile brain-body co-adaptation and thus promote MAP-Elites migrations. Pollination increases the success of body mutations and the number of migrations, resulting in better quality-diversity metrics. We believe we develop important insights that could apply to other domains where MAP-Elites is used.

Figures (10)

• Figure 1: Applying MAP-Elites algorithm directly, Standard treatment, results in seemingly reasonable performance. Fitness values for each niche at the end of the run averaged over 20 repetitions (left) and the fitness of the best individual during evolutionary time (middle) shows reasonable performance. Inspecting the number of migrations, however, shows a trend where the number of migrations quickly decreases. This observation raises concerns given that the strength of the MAP-Elites algorithm comes from its ability to create stepping stones. The solid lines show the averages over 20 repetitions, and the shaded regions show a 95% confidence interval.
• Figure 2: Archive of a selected run and the lineage of the run champion plotted on the niche space, where arrows show migration events (left). Fitness and generation of each migrating solution (right). While it seems like MAP-Elites is creating many stepping stones, these events happen at a low fitness regime and very early on in the evolution. Please note the scale on the y-axis.
• Figure 3: Comparison of quality metrics for the Standard and No Migration treatments. Standard treatment consistently fills up all of the available niches while the No Migration treatment loses the morphological diversity as there is no explicit mechanism to protect it, which results in better collection size and total quality for the Standard treatment. However, both treatments perform comparably in maximal quality, prompting us to question whether we could have found better solutions with more migrations. Horizontal lines shows the results of statistical tests. *: $P<0.05$, n.s.: $P>=0.05$
• Figure 4: Distribution of the average relative fitness of the offspring created through body mutations relative to their parents at each repetition of the Standard and Pollination treatments. Pollination Regular covers the offspring with no pollinated ancestor -- none of their ancestors have had their controllers replaced with a distilled controller. Pollination Pollinated covers the offspring with at least one pollinated ancestor in their lineage. The distributions of the average relative fitness for the Standard treatment and the regular offspring in the Pollination treatment are statistically indistinguishable at level $p>0.05$. On the other hand, offspring with pollinated ancestors are more robust to body mutations compared to other groups, confirming that distilled controllers transfer better to other morphologies and can potentially encourage more migrations.
• Figure 5: We investigate the migrations occurred in the Standard and Pollination treatments. Migrations over evolutionary time (left) and distributions of the average number of migrations per generation for each repetition (middle) are plotted. While both treatments show a similar trend over evolutionary time, Pollination treatment is capable of producing more migrations on average during whole evolution and on average in total. The total number of migrations is plotted against the number of migrations where the migrating solution has at least one pollinated ancestor, for each repetition of the Pollination treatment (right). The strong correlation shows that the migration increase results from the proposed pollination procedure.