G-EvoNAS: Evolutionary Neural Architecture Search Based on Network Growth

TL;DR

G-EvoNAS tackles the challenge of efficiently searching the global neural architecture space by introducing a growth-based evolutionary strategy that progressively deepens network Blocks. It combines a block-wise search space with staged evolutionary growth and phasic SuperNet pruning to accelerate evaluation and enhance ranking fidelity, achieving competitive CIFAR-10/100 results in just $0.2$ GPU days and transferring effectively to ImageNet with strong top-1 performance. The key contributions are the block-based global search space, the growth-oriented evolutionary search, and the pruning-driven refinement of the SuperNet, which together drastically reduce search cost while preserving or improving accuracy. This approach enables rapid discovery of high-performing architectures suitable for transfer learning, with practical implications for efficient NAS in resource-constrained settings.

Abstract

The evolutionary paradigm has been successfully applied to neural network search(NAS) in recent years. Due to the vast search complexity of the global space, current research mainly seeks to repeatedly stack partial architectures to build the entire model or to seek the entire model based on manually designed benchmark modules. The above two methods are attempts to reduce the search difficulty by narrowing the search space. To efficiently search network architecture in the global space, this paper proposes another solution, namely a computationally efficient neural architecture evolutionary search framework based on network growth (G-EvoNAS). The complete network is obtained by gradually deepening different Blocks. The process begins from a shallow network, grows and evolves, and gradually deepens into a complete network, reducing the search complexity in the global space. Then, to improve the ranking accuracy of the network, we reduce the weight coupling of each network in the SuperNet by pruning the SuperNet according to elite groups at different growth stages. The G-EvoNAS is tested on three commonly used image classification datasets, CIFAR10, CIFAR100, and ImageNet, and compared with various state-of-the-art algorithms, including hand-designed networks and NAS networks. Experimental results demonstrate that G-EvoNAS can find a neural network architecture comparable to state-of-the-art designs in 0.2 GPU days.

Figures (4)

• Figure 1: Overall framework of G-EvoNAS.(a) Evolutionary search process. During the search process of population evolution, only the latest growing Block will be searched. (b) Network growth process, the network length will increase by one Block at each growth stage. (c) Supernet pruning, the SuperNet corresponding to the current growth stage will be searched. The block in is pruned. Figure (c) shows an example of pruning two blocks.
• Figure 2: Top: A design network architecture consisting of 8 blocks. Middle and bottom: Block 2 and its block code example. A block consists of 4 hidden nodes, 2 source nodes (node 0 and node 1) and one output node (node 6). Right side: nodes represent two inputs that are given an output by an add operation after the corresponding operation.
• Figure 3: (a)Hyperparametric analysis on CIFAR10. We chose population sizes of 10,15, and 20 and number of growth phase iterations of 10, 30, and 50. (b)Comparison of model rankings for G-EvoNAS (left) and One-Shot (right).
• Figure 4: Comparing the effectiveness of G-EvoNAS using network growing and SuperNet pruning on CIFAR10 and CIFAR100. In (a) and (b), the orange line shows the best fitness in each generation during the evolution iteration. The blue line marks the best validation accuracy of the models that were sampled by the global search in each iteration of the parameter sharing method. In (c) and (d), we plot the validation accuracy of the two networks found by both methods during the training. Each network is trained for 600 epochs.