Exploring Deep-to-Shallow Transformable Neural Networks for Intelligent Embedded Systems

TL;DR

DW-NAS introduces a novel deep-to-shallow transformable NAS framework to resolve the accuracy–efficiency dilemma in embedded CNNs by first maximizing accuracy with deep networks and then transforming them into shallow, hardware-friendly equivalents without accuracy loss. It combines a hybrid transformable search space, a vanilla latency predictor, and a sandwich-inspired differentiable search to efficiently navigate around a latency constraint, along with hybrid transformable and arbitrary-resolution elastic training to enable runtime elasticity. The approach is validated on NVIDIA Jetson Xavier/Nano and large-scale datasets (ImageNet, ImageNet-100), showing superior accuracy–latency trade-offs compared with state-of-the-art HW-DNAS methods and robust ablations demonstrating component contributions. The work also provides a practical path to elastic inference across arbitrary input resolutions, reducing training and storage overhead while maintaining on-device efficiency, with implications for robust, energy-aware embedded AI deployment.

Abstract

Thanks to the evolving network depth, convolutional neural networks (CNNs) have achieved remarkable success across various embedded scenarios, paving the way for ubiquitous embedded intelligence. Despite its promise, the evolving network depth comes at the cost of degraded hardware efficiency. In contrast to deep networks, shallow networks can deliver superior hardware efficiency but often suffer from inferior accuracy. To address this dilemma, we propose Double-Win NAS, a novel deep-to-shallow transformable neural architecture search (NAS) paradigm tailored for resource-constrained intelligent embedded systems. Specifically, Double-Win NAS strives to automatically explore deep networks to first win strong accuracy, which are then equivalently transformed into their shallow counterparts to further win strong hardware efficiency. In addition to search, we also propose two enhanced training techniques, including hybrid transformable training towards better training accuracy and arbitrary-resolution elastic training towards enabling natural network elasticity across arbitrary input resolutions. Extensive experimental results on two popular intelligent embedded systems (i.e., NVIDIA Jetson AGX Xavier and NVIDIA Jetson Nano) and two representative large-scale datasets (i.e., ImageNet and ImageNet-100) clearly demonstrate the superiority of Double-Win NAS over previous state-of-the-art NAS approaches.

Figures (22)

• Figure 1: An intuitive overview of Double-Win NAS (DW-NAS) that focuses on exploring and training deep-to-shallow transformable networks.
• Figure 2: The layer-wise latency profiling analysis of MobileNetV2 on Xavier (left) and Nano (right), where only ReLU6 is non-linear. The reported percentage corresponds to the latency percentage of different layers, compared with the total latency of MobileNetV2.
• Figure 3: Comparisons of shallow and narrow networks on Xavier (left) and Nano (right). Note that the experimental settings are similar to luo2024pearls. The blue and orange nodes correspond to the speedups of shallow and narrow networks derived from layer-wise pruning and channel-wise pruning, compared with the original MobileNetV2.
• Figure 4: Architecture search results under $\lambda \in [0, 1]$ on Xavier (left) and Nano (right), in which the latency is measured with an input batch size of 8 and the accuracy is trained on ImageNet for 50 epochs.
• Figure 5: Comparisons of the accuracy of MobileNetV2 on ImageNet under elastic and stand-alone settings across diverse input resolutions, where the latency is measured on Xavier (left) and Nano (right).