Meta knowledge assisted Evolutionary Neural Architecture Search

TL;DR

The paper tackles the high computational cost of evolutionary NAS and the information loss from fixed learning-rate schedules. It introduces MetaNAS, a meta-knowledge–assisted EC-NAS framework that learns a dynamic LR schedule ($Meta-LR$) from training loss, uses an adaptive surrogate model with a threshold to screen architectures, and employs a period mutation operator to boost diversity. Together, these components enable efficient, robust search and produce competitive architectures on CIFAR-10/100 and ImageNet1K with substantially reduced search cost. The approach demonstrates strong generalization across architectures and datasets while maintaining competitive accuracy and resource usage, highlighting practical benefits for resource-constrained NAS deployments.

Abstract

Evolutionary computation (EC)-based neural architecture search (NAS) has achieved remarkable performance in the automatic design of neural architectures. However, the high computational cost associated with evaluating searched architectures poses a challenge for these methods, and a fixed form of learning rate (LR) schedule means greater information loss on diverse searched architectures. This paper introduces an efficient EC-based NAS method to solve these problems via an innovative meta-learning framework. Specifically, a meta-learning-rate (Meta-LR) scheme is used through pretraining to obtain a suitable LR schedule, which guides the training process with lower information loss when evaluating each individual. An adaptive surrogate model is designed through an adaptive threshold to select the potential architectures in a few epochs and then evaluate the potential architectures with complete epochs. Additionally, a periodic mutation operator is proposed to increase the diversity of the population, which enhances the generalizability and robustness. Experiments on CIFAR-10, CIFAR-100, and ImageNet1K datasets demonstrate that the proposed method achieves high performance comparable to that of many state-of-the-art peer methods, with lower computational cost and greater robustness.

Figures (8)

• Figure 1: Overall diagram of the proposed MetaNAS, which combines six steps. We first obtain the individuals by initializing the population. The proposed MetaNAS framework is subsequently used to find the individuals with the best fitness to decode the best architecture.
• Figure 2: Computational graph of Meta-LR in MetaNAS.
• Figure 3: Correlation between the validation error rate and the number of parameters in the architectures on CIFAR-10.
• Figure 4: Search space of the proposed MetaNAS.
• Figure 5: Optimal architectures obtained by MetaNAS on CIFAR-10 and CIFAR-100.