Growing Artificial Neural Networks for Control: the Role of Neuronal Diversity

TL;DR

This work addresses how to grow artificial neural networks (ANNs) by mimicking biological growth, a process that faces stability challenges when diversity is not maintained. It introduces Neural Developmental Programs (NDPs) with two diversity-preserving mechanisms: intrinsic hidden states inherited during neurogenesis and lateral inhibition that slows concurrent growth actions. Empirical results show that NDPs with these mechanisms achieve performance comparable to indirect encodings and can approach direct-encoded baselines on complex locomotion tasks, while ablations lacking intrinsic states fail. The approach lays groundwork for scalable, robust growth-driven control in high-dimensional reinforcement learning settings, suggesting that diversity-aware growth can unlock more generalizable and adaptable policies.

Abstract

In biological evolution complex neural structures grow from a handful of cellular ingredients. As genomes in nature are bounded in size, this complexity is achieved by a growth process where cells communicate locally to decide whether to differentiate, proliferate and connect with other cells. This self-organisation is hypothesized to play an important part in the generalisation, and robustness of biological neural networks. Artificial neural networks (ANNs), on the other hand, are traditionally optimized in the space of weights. Thus, the benefits and challenges of growing artificial neural networks remain understudied. Building on the previously introduced Neural Developmental Programs (NDP), in this work we present an algorithm for growing ANNs that solve reinforcement learning tasks. We identify a key challenge: ensuring phenotypic complexity requires maintaining neuronal diversity, but this diversity comes at the cost of optimization stability. To address this, we introduce two mechanisms: (a) equipping neurons with an intrinsic state inherited upon neurogenesis; (b) lateral inhibition, a mechanism inspired by biological growth, which controlls the pace of growth, helping diversity persist. We show that both mechanisms contribute to neuronal diversity and that, equipped with them, NDPs achieve comparable results to existing direct and developmental encodings in complex locomotion tasks

Figures (2)

• Figure 1: Training curves for different control tasks and methods. NDPs perform comparably to the other indirect encoding.
• Figure 2: Effect of inhibition on neuronal diversity in the Reacher during evolution (left) and during growth (right).