Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ecological Neural Architecture Search

Benjamin David Winter, William J. Teahan

TL;DR

ENAS tackles the NAS hyperparameter tuning problem by evolving evolutionary parameters alongside architecture traits, encoding four ecological genes into each candidate's genome. It dynamically adjusts population size, mutation rate, cloning rate, and max generations based on fitness, enabling self-regulation and reducing wasted compute. Across four binary classification datasets, ENAS achieves higher average fitness across all datasets and higher maximum fitness in 3 of 4 datasets, with a time reduction of $18.3\%$ and dataset-specific architectural preferences emerging. The approach promises automated hyperparameter control and efficiency gains, with potential extensions to more complex architectures and multi-objective optimization.

Abstract

When employing an evolutionary algorithm to optimize a neural networks architecture, developers face the added challenge of tuning the evolutionary algorithm's own hyperparameters - population size, mutation rate, cloning rate, and number of generations. This paper introduces Neuvo Ecological Neural Architecture Search (ENAS), a novel method that incorporates these evolutionary parameters directly into the candidate solutions' phenotypes, allowing them to evolve dynamically alongside architecture specifications. Experimental results across four binary classification datasets demonstrate that ENAS not only eliminates manual tuning of evolutionary parameters but also outperforms competitor NAS methodologies in convergence speed (reducing computational time by 18.3%) and accuracy (improving classification performance in 3 out of 4 datasets). By enabling "greedy individuals" to optimize resource allocation based on fitness, ENAS provides an efficient, self-regulating approach to neural architecture search.

Ecological Neural Architecture Search

TL;DR

ENAS tackles the NAS hyperparameter tuning problem by evolving evolutionary parameters alongside architecture traits, encoding four ecological genes into each candidate's genome. It dynamically adjusts population size, mutation rate, cloning rate, and max generations based on fitness, enabling self-regulation and reducing wasted compute. Across four binary classification datasets, ENAS achieves higher average fitness across all datasets and higher maximum fitness in 3 of 4 datasets, with a time reduction of and dataset-specific architectural preferences emerging. The approach promises automated hyperparameter control and efficiency gains, with potential extensions to more complex architectures and multi-objective optimization.

Abstract

When employing an evolutionary algorithm to optimize a neural networks architecture, developers face the added challenge of tuning the evolutionary algorithm's own hyperparameters - population size, mutation rate, cloning rate, and number of generations. This paper introduces Neuvo Ecological Neural Architecture Search (ENAS), a novel method that incorporates these evolutionary parameters directly into the candidate solutions' phenotypes, allowing them to evolve dynamically alongside architecture specifications. Experimental results across four binary classification datasets demonstrate that ENAS not only eliminates manual tuning of evolutionary parameters but also outperforms competitor NAS methodologies in convergence speed (reducing computational time by 18.3%) and accuracy (improving classification performance in 3 out of 4 datasets). By enabling "greedy individuals" to optimize resource allocation based on fitness, ENAS provides an efficient, self-regulating approach to neural architecture search.

Paper Structure

This paper contains 10 sections, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Methodology
  4. Experimental Setup
  5. Datasets
  6. Experimental Results and Discussion
  7. Evolutionary Parameter Convergence
  8. Architecture Convergence
  9. Computational Efficiency
  10. Conclusion and Future Work

Figures (3)

  • Figure 1: A Christmas Island red crab devouring its own offspring, illustrating natural resource optimization at the expense of population diversity.
  • Figure 2: Evolution of average population fitness and mutation rate for the Heart dataset, showing how mutation rate dynamically adjusts based on fitness improvements.
  • Figure 3: Evolution of mean population fitness and mutation rate for the Sonar dataset, demonstrating gradual enhancements to the globally optimal model.