Meta-Learning an Evolvable Developmental Encoding

TL;DR

The paper tackles learning representations for black-box optimisation by meta-learning a DNA-guided developmental encoding implemented as a Neural Cellular Automaton. The method uses a quality-diversity objective with MAP-Elites in the outer loop and DNA-driven development in the inner loop to evolve a genotype-to-phenotype mapping that increases search speed and output diversity. The key contributions are showing evolvable genotype-to-phenotype mappings that produce diverse, high-quality mazes, and providing diagnostic analyses of DNA usage and design decisions. This work advances autonomous, evolvable representations for optimization and design tasks, with potential implications for more robust and adaptable AI systems.

Abstract

Representations for black-box optimisation methods (such as evolutionary algorithms) are traditionally constructed using a delicate manual process. This is in contrast to the representation that maps DNAs to phenotypes in biological organisms, which is at the hear of biological complexity and evolvability. Additionally, the core of this process is fundamentally the same across nearly all forms of life, reflecting their shared evolutionary origin. Generative models have shown promise in being learnable representations for black-box optimisation but they are not per se designed to be easily searchable. Here we present a system that can meta-learn such representation by directly optimising for a representation's ability to generate quality-diversity. In more detail, we show our meta-learning approach can find one Neural Cellular Automata, in which cells can attend to different parts of a "DNA" string genome during development, enabling it to grow different solvable 2D maze structures. We show that the evolved genotype-to-phenotype mappings become more and more evolvable, not only resulting in a faster search but also increasing the quality and diversity of grown artefacts.

Figures (6)

• Figure 1: Meta-learning DNA-like representations. (a) The sketch of DNA-guided development. A component $i$ of a developmental system with some hidden state $\mathbf{h}_i^t$ sends and receives information from other components in the system (blue arrows). Additionally, the DNA encoding (gray) provides a global set of instructions via state-dependent decoding mechanism (red). The decoding mechanism produces an output $\mathbf{c}_i^t$ which conditions the state vector (plus sign) producing a vector $\mathbf{u}_i^t$ which is then used for any downstream operation in the node (e.g. determining which information to send to other components). (b) General procedure to train a parameterized developmental system $S$ (e.g. NCA). An outer loop evolutionary strategy generates a population of possible parameters for $S$. Each of these (blue) is evaluated (inside the red square) on their capacity to fill an archive of solutions (in this example, simple mazes) defined according to their quality and diversity values. The developmental system must achieve this by maximizing its usage of the potential structure in the DNA-like encodings (red).
• Figure 2: Progression of QD metrics across training. Left panel: the mean fitness scores (y-axis) for solutions in an archive at each step of an inner loop run (x-axis) at different points during training (different colored curves). Middle panel: Same but for the coverage. Right: The quality ratio at each time-step defined as the average quality of the solutions in the archive relative to the maximum possible. These results demonstrate the evolvability of the NCAs. Early in evolution, NCAs produce neither high coverage nor high-quality solutions. After the NCA training (i.e. the genotype-to-phenotype mapping), DNA genomes can quickly be evolved that result in high-quality and diverse phenotypes. Note that the first and last panel are similar, but complement each other since the latter tracks the maximum performance that could have been achieved without increasing coverage.
• Figure 3: Repertoires during inner loop exploration of the encoding space. a) Each panel shows a different state of the archive during an inner loop run for a trained model. Yellow represents higher-quality solutions and purple low-quality ones as defined by the number of connected components in the maze, with lower being better. b) Example mazes all grown by the same NCA shown together with the associated DNA sequences. The model succeeds at covering almost half of the descriptor space with high-quality solutions. These have different sizes and shapes, showing that the model is effective at controlling its growth. Some similarities between the DNAs can be observed, especially in the top row examples.
• Figure 4: Growing mazes using DNA-guided NCAs. Each column in the top row shows a different step in the models development of a maze, from step 1 to 50. The bottom row shows the position of the DNA sequence with the highest attention weight for each cell in the NCA at the corresponding time step. Dark cells represent cells that are not yet alive. The attention maps show that these follow a similar pattern as the resulting shape, pointing to how each position is likely being used by the system to grow the corresponding shape.
• Figure 5: Measuring the structure of the DNA representation. The values for three metrics which measure potential structure that appears in DNA-space as training of the developmental system progresses. This indicates that there is emergent structure in the genotype space as a result of training, but that the model is likely susceptible to becoming chaotic as this can be an effective strategy for covering the behavioral space.