A Multi-objective Evolutionary Algorithm Based on Bi-population with Uniform Sampling for Neural Architecture Search

TL;DR

The paper tackles the challenge of multi-objective neural architecture search by introducing MOEA-BUS, a bi-population evolutionary algorithm with uniform sampling that targets accuracy and MAdds. It combines a uniform-initialization strategy with two evolving populations that exchange elites, aided by a surrogate model and weight inheritance to cut evaluation cost. Empirical results on CIFAR-10 and ImageNet show MOEA-BUS achieves superior accuracy–complexity trade-offs with low search cost, and extensive ablations confirm the value of both uniform sampling and the bi-population framework. The approach demonstrates strong Pareto-front coverage and diversity, suggesting practical benefits for efficient NAS under real-world resource constraints.

Abstract

Neural architecture search (NAS) automates neural network design, improving efficiency over manual approaches. However, efficiently discovering high-performance neural network architectures that simultaneously optimize multiple objectives remains a significant challenge in NAS. Existing methods often suffer from limited population diversity and inadequate exploration of the search space, particularly in regions with extreme complexity values. To address these challenges, we propose MOEA-BUS, an innovative multi-objective evolutionary algorithm based on bi-population with uniform sampling for neural architecture search, aimed at simultaneously optimizing both accuracy and network complexity. In MOEA-BUS, a novel uniform sampling method is proposed to initialize the population, ensuring that architectures are distributed uniformly across the objective space. Furthermore, to enhance exploration, we deploy a bi-population framework where two populations evolve synergistically, facilitating comprehensive search space coverage. Experiments on CIFAR-10 and ImageNet demonstrate MOEA-BUS's superiority, achieving top-1 accuracies of 98.39% on CIFAR-10, and 80.03% on ImageNet. Notably, it achieves 78.28% accuracy on ImageNet with only 446M MAdds. Ablation studies confirm that both uniform sampling and bi-population mechanisms enhance population diversity and performance. Additionally, in terms of the Kendall's tau coefficient, the SVM achieves an improvement of at least 0.035 compared to the other three commonly used machine learning models, and uniform sampling provided an enhancement of approximately 0.07.

Figures (9)

• Figure 1: Overall Framework: A multi-objective evolutionary neural architecture search method based on bi-population with uniform sampling.
• Figure 2: Search space and encoding. (a) The architecture search space. (b) An example of the encoding. The encoding is divided into five parts by blocks. The parameters we search include image resolution, the number of layers in each block, the expansion rate, and the kernel size in each layer.
• Figure 3: The distribution of randomly sampled 5,000 architectures.
• Figure 4: The illustration of uniform sampling.
• Figure 5: The illustration of the proposed surrogate model.