NEVO-GSPT: Population-Based Neural Network Evolution Using Inflate and Deflate Operators

TL;DR

This work tackles the high cost and opaque mapping between architecture and behavior in neural architecture search by introducing NEVO-GSPT, a population-based Neuroevolution method that leverages Geometric Semantic Genetic Programming principles. It introduces Inflate and Deflate Mutations (IGSM and DGSM) to control growth and maintain a unimodal fitness landscape, while enabling efficient evaluation through a linked-list representation of perturbation components. The method achieves competitive regression performance with notably smaller networks across four benchmarks and demonstrates substantial CPU-based efficiency, enabling population-level exploration at minutes instead of GPU-days. These contributions advance interpretable, scalable neural architecture search by coupling gradient-based optimization with principled geometric semantics. The findings suggest broad applicability in resource-constrained settings where model interpretability and efficiency are crucial.

Abstract

Evolving neural network architectures is a computationally demanding process. Traditional methods often require an extensive search through large architectural spaces and offer limited understanding of how structural modifications influence model behavior. This paper introduces \gls{ngspt}, a novel Neuroevolution algorithm based on two key innovations. First, we adapt geometric semantic operators~(GSOs) from genetic programming to neural network evolution, ensuring that architectural changes produce predictable effects on network semantics within a unimodal error surface. Second, we introduce a novel operator (DGSM) that enables controlled reduction of network size, while maintaining the semantic properties of~GSOs. Unlike traditional approaches, \gls{ngspt}'s efficient evaluation mechanism, which only requires computing the semantics of newly added components, allows for efficient population-based training, resulting in a comprehensive exploration of the search space at a fraction of the computational cost. Experimental results on four regression benchmarks show that \gls{ngspt} consistently evolves compact neural networks that achieve performance comparable to or better than established methods in the literature, such as standard neural networks, SLIM-GSGP, TensorNEAT, and SLM.

Figures (9)

• Figure 1: Illustration of using both neural network and linked list representations.
• Figure 2: Illustration of using both neural network and linked list representations.
• Figure 3: Comparison of the impact of AprT on the evolutionary process, showing RMSE performance on the training and test sets across all datasets.
• Figure 4: Comparison of the impact of ApoT on the evolutionary process, showing RMSE performance on the test set across all datasets.
• Figure 5: Comparing the RMSE performance of difference combinations of inflate and deflate probabilities.