Selection for short-term empowerment accelerates the evolution of homeostatic neural cellular automata

TL;DR

The paper investigates how the time horizon used in calculating empowerment affects the evolution of neural cellular automata (NCA) tasked with morphogenesis and homeostasis. Using Age Fitness Pareto Optimization (AFPO) with objectives for age, loss, and empowerment, specifically the multi-agent empowerment $\mathfrak{E}(k) = I(A_0^{k}, S_{N-k}^{N})$, the authors compare short ($k=1$) versus long ($k$ up to 45) horizons and find that shorter horizons yield stronger, more stable, and more generalizable NCAs. Short-term empowerment produces cohesive shapes, longer-term stability after unseen perturbations, and better transfer to new morphogenesis targets, suggesting a pre-training-like advantage. The findings highlight time-scale as a critical design factor for universal objective functions and have implications for accelerating evolution of robust, self-organizing artificial systems; code is available at the provided GitHub repository.

Abstract

Empowerment -- a domain independent, information-theoretic metric -- has previously been shown to assist in the evolutionary search for neural cellular automata (NCA) capable of homeostasis when employed as a fitness function. In our previous study, we successfully extended empowerment, defined as maximum time-lagged mutual information between agents' actions and future sensations, to a distributed sensorimotor system embodied as an NCA. However, the time-delay between actions and their corresponding sensations was arbitrarily chosen. Here, we expand upon previous work by exploring how the time scale at which empowerment operates impacts its efficacy as an auxiliary objective to accelerate the discovery of homeostatic NCAs. We show that shorter time delays result in marked improvements over empowerment with longer delays, when compared to evolutionary selection only for homeostasis. Moreover, we evaluate stability and adaptability of evolved NCAs, both hallmarks of living systems that are of interest to replicate in artificial ones. We find that short-term empowered NCA are more stable and are capable of generalizing better to unseen homeostatic challenges. Taken together, these findings motivate the use of empowerment during the evolution of other artifacts, and suggest how it should be incorporated to accelerate evolution of desired behaviors for them. Source code for the experiments in this paper can be found at: https://github.com/caitlingrasso/empowered-nca-II.

Figures (8)

• Figure 1: Construction of sensorimotor time series for empowerment calculation with different time horizons, $k$. At each time-step $n$ during the NCA simulation (A) the neural network (B) is executed for each live cell $i,j$ to produce the state of the NCA at the following time-step $n+1$ (C). Neural network inputs and outputs define the sensor state $s_{ijn}$ (D) and action state $a_{ijn}$ (E), respectively, based on the signaling channel associated with a given cell and time-step. These values are collected over the total length of simulation, $N$, to construct action and sensor time series for cell $i,j$ (F,G). The sensor time series are shifted according to the time horizon $k$ to produce a set of sensor/action pairs (unshaded boxes). Shaded boxes are not included in the empowerment calculation due to the value of the time horizon which can either be short (F), long (G) or somewhere in between.
• Figure 2: (A) Loss of the best NCA at the end of 2000 generations averaged over all replicates (with 95% confidence intervals) for 10 different evolutionary variants. $\dagger$ indicates a significant difference from the bi-loss control and $\ddagger$ indicates a significant different from the tri-loss control with a Bonferroni-corrected significance level of $p<0.0031$. (B) Average loss (with 95% confidence intervals) over evolutionary time for both controls and empowerment runs computed with a time-horizon of $k=1$. The tri-loss-empowerment curve (green) is significantly lower that the loss-only control curves (blue and orange). This gap lessens with an increase in time-horizon as seen with $k=25$ (B) and $k=45$ (C).
• Figure 3: Average loss at time $n$ during NCA simulation of the most fit NCA from each evolutionary trial with 95% confidence intervals. NCA were run 50 iterations past the simulation time-length during evolution to assess long-term stability. Linear regressions were performed on the average loss curves with the slope $m$ displayed to the right.
• Figure 4: NCA evolved for short-term ($k=1$) empowerment display behaviors that are favorable for the shape-matching task including low long-term instability (A), low cell transiency (B), and cohesive shapes consisting of few connected components (C) compared to loss-only controls and long-term ($k=45$) empowered NCA. There is, however, no significant difference in proportion of cells that run up against the boundary at the end of simulation. Error bars indicate 95% confidence intervals.
• Figure 5: Evolved NCA from the loss-only trials (blue and orange curves) or from the short-term empowered trials (purple and green curves) were used to seed initial populations of evolutionary runs for three different target shapes: a triangle (A), an X (B), and a soft biped shape (C). The blue, orange, and purple curves (evolution seeded with short-term empowered NCA) are evolved for bi-loss and the green curve is evolved for tri-loss-empowerment with $k=1$. The best NCA from the bi-loss runs seeded with short-term empowered NCA are displayed to the right where the green outline indicates the target shape, magenta indicates the shape of the NCA at the end of development, and the grayscale represents the signaling channel.