Single Path One-Shot Neural Architecture Search with Uniform Sampling
Zichao Guo, Xiangyu Zhang, Haoyuan Mu, Wen Heng, Zechun Liu, Yichen Wei, Jian Sun
TL;DR
This work addresses the high cost and optimization bias of NAS on large datasets by proposing a single-path one-shot framework trained with uniform path sampling, effectively decoupling weights and enabling efficient architecture search under hard constraints. It introduces novel choice blocks for channel number and mixed-precision quantization, and uses evolutionary search to find architectures that satisfy latency/FLOPs constraints. Extensive ImageNet experiments demonstrate state-of-the-art accuracy with lower training and search costs, and show the method's flexibility across search spaces and hardware constraints. A NAS-Bench correlation study indicates partial but informative ranking alignment between the supernet and final models, highlighting both the practicality and current limits of the approach.
Abstract
We revisit the one-shot Neural Architecture Search (NAS) paradigm and analyze its advantages over existing NAS approaches. Existing one-shot method, however, is hard to train and not yet effective on large scale datasets like ImageNet. This work propose a Single Path One-Shot model to address the challenge in the training. Our central idea is to construct a simplified supernet, where all architectures are single paths so that weight co-adaption problem is alleviated. Training is performed by uniform path sampling. All architectures (and their weights) are trained fully and equally. Comprehensive experiments verify that our approach is flexible and effective. It is easy to train and fast to search. It effortlessly supports complex search spaces (e.g., building blocks, channel, mixed-precision quantization) and different search constraints (e.g., FLOPs, latency). It is thus convenient to use for various needs. It achieves start-of-the-art performance on the large dataset ImageNet.