Searching Search Spaces: Meta-evolving a Geometric Encoding for Neural Networks

TL;DR

The paper tackles policy search for neural networks by leveraging an indirect, geometry-based encoding (GENE) whose distance function governs connection weights. It introduces meta-evolution with Cartesian Genetic Programming to automatically learn effective distance functions, yielding learned encodings that outperform direct encoding and hand-crafted distances on several continuous-control tasks and even generalize to new environments. Key findings include the emergence of sparse networks via pruning-like distance rules, and a simple, interpretable best function LD-367 that uses a boolean comparison to selectively propagate weights. The work demonstrates that evolving the encoding itself can meaningfully reshape the search landscape, enabling more efficient and scalable neuroevolution with potential for broader applicability and exploration of sparse representations.

Abstract

In evolutionary policy search, neural networks are usually represented using a direct mapping: each gene encodes one network weight. Indirect encoding methods, where each gene can encode for multiple weights, shorten the genome to reduce the dimensions of the search space and better exploit permutations and symmetries. The Geometric Encoding for Neural network Evolution (GENE) introduced an indirect encoding where the weight of a connection is computed as the (pseudo-)distance between the two linked neurons, leading to a genome size growing linearly with the number of genes instead of quadratically in direct encoding. However GENE still relies on hand-crafted distance functions with no prior optimization. Here we show that better performing distance functions can be found for GENE using Cartesian Genetic Programming (CGP) in a meta-evolution approach, hence optimizing the encoding to create a search space that is easier to exploit. We show that GENE with a learned function can outperform both direct encoding and the hand-crafted distances, generalizing on unseen problems, and we study how the encoding impacts neural network properties.

Figures (8)

• Figure 1: Meta-evolution process, depicting the different steps happening. In orange are the evaluation steps conducted in parallel. Left: The outer loop is a CGP process, generating distance functions and evaluating them. Center: The components of distance function evaluation. The final fitness used for the CGP outer-loop is $F=\beta f_\text{task} - (1-\beta)f_\text{prop} + \alpha f_\text{input}$. Right: The distance function being evaluated is used as GENE encoding in a loop to optimize neural networks with and ES. This operation is done on both HalfCheetah and Walker2D with $f_\text{task}$ the sum of normalized final fitness scores.
• Figure 2: Visualization of the CGP graph of the learned distance function. In red are the input coordinates of the first neuron and in orange are the input values of the second neuron. In blue are the operator nodes, used by CGP to construct a function. The output node is depicted in green. The function can be simplified as $d=(x_2>z_1)y_2$, as $>$ is a boolean operator. In total, 3 input nodes, 3 operator nodes and the single output node are used.
• Figure 3: Weight values distribution for neural networks trained with direct encoding and GENE with pl2 and learned distances.
• Figure 4: Maximum fitness reached by an ES with each encoding, averaged over $5$ runs.
• Figure 5: Fitness of the center individual of the population, averaged over $5$ run. The noisiness can be explained by the fact that the mean of the population is not very stable compared to the best observed individual.