GreenMachine: Automatic Design of Zero-Cost Proxies for Energy-Efficient NAS

TL;DR

This paper addresses the challenges of model evaluation by automatically designing zero-cost proxies to assess DNNs efficiently, and assess the proxies' effectiveness using both randomly sampled and stratified subsets of the search space, ensuring they can differentiate between low- and high-performing networks and enhance generalizability.

Abstract

Artificial Intelligence (AI) has driven innovations and created new opportunities across various sectors. However, leveraging domain-specific knowledge often requires automated tools to design and configure models effectively. In the case of Deep Neural Networks (DNNs), researchers and practitioners usually resort to Neural Architecture Search (NAS) approaches, which are resource- and time-intensive, requiring the training and evaluation of numerous candidate architectures. This raises sustainability concerns, particularly due to the high energy demands involved, creating a paradox: the pursuit of the most effective model can undermine sustainability goals. To mitigate this issue, zero-cost proxies have emerged as a promising alternative. These proxies estimate a model's performance without the need for full training, offering a more efficient approach. This paper addresses the challenges of model evaluation by automatically designing zero-cost proxies to assess DNNs efficiently. Our method begins with a randomly generated set of zero-cost proxies, which are evolved and tested using the NATS-Bench benchmark. We assess the proxies' effectiveness using both randomly sampled and stratified subsets of the search space, ensuring they can differentiate between low- and high-performing networks and enhance generalizability. Results show our method outperforms existing approaches on the stratified sampling strategy, achieving strong correlations with ground truth performance, including a Kendall correlation of 0.89 on CIFAR-10 and 0.77 on CIFAR-100 with NATS-Bench-SSS and a Kendall correlation of 0.78 on CIFAR-10 and 0.71 on CIFAR-100 with NATS-Bench-TSS.

Figures (7)

• Figure 1: Simplified visualization of the flow of an .
• Figure 2: Example of a grammar.
• Figure 3: Example of Grammatical Evolution mapping.
• Figure 4: Histograms of test accuracy for the NATS-Bench benchmark across the topology and size search spaces and the CIFAR-10, CIFAR-100, and ImageNet16-120 datasets.
• Figure 5: Evolution of the Kendall correlation coefficient on NATS-Bench's two search spaces and the fitness metric over 100 generations, averaged over 5 runs.