DeepFedNAS: A Unified Framework for Principled, Hardware-Aware, and Predictor-Free Federated Neural Architecture Search

TL;DR

DeepFedNAS tackles the bottlenecks in Federated Neural Architecture Search by introducing a principled, multi-objective fitness function $\mathcal{F}(\mathcal{A})$ and a Pareto-path curriculum to train a re-engineered ResNet-based supernet. It then eliminates the costly accuracy predictor by leveraging $\mathcal{F}(\mathcal{A})$ as a zero-cost proxy for subnet performance, enabling rapid, on-demand subnet discovery under hardware constraints. The framework achieves state-of-the-art accuracy on image datasets, reduces post-training search time by approximately $61\times$, and demonstrates strong robustness to non-IID data and varying client participation, while supporting latency and parameter budgets for real-world deployment. Collectively, DeepFedNAS makes hardware-aware FL deployments practical and scalable by integrating principled design, efficient training, and predictor-free search into a single end-to-end pipeline.

Abstract

Federated Neural Architecture Search (FedNAS) aims to automate model design for privacy-preserving Federated Learning (FL) but currently faces two critical bottlenecks: unguided supernet training that yields suboptimal models, and costly multi-hour pipelines for post-training subnet discovery. We introduce DeepFedNAS, a novel, two-phase framework underpinned by a principled, multi-objective fitness function that synthesizes mathematical network design with architectural heuristics. Enabled by a re-engineered supernet, DeepFedNAS introduces Federated Pareto Optimal Supernet Training, which leverages a pre-computed Pareto-optimal cache of high-fitness architectures as an intelligent curriculum to optimize shared supernet weights. Subsequently, its Predictor-Free Search Method eliminates the need for costly accuracy surrogates by utilizing this fitness function as a direct, zero-cost proxy for accuracy, enabling on-demand subnet discovery in mere seconds. DeepFedNAS achieves state-of-the-art accuracy (e.g., up to 1.21% absolute improvement on CIFAR-100), superior parameter and communication efficiency, and a substantial ~61x speedup in total post-training search pipeline time. By reducing the pipeline from over 20 hours to approximately 20 minutes (including initial cache generation) and enabling 20-second individual subnet searches, DeepFedNAS makes hardware-aware FL deployments instantaneous and practical. The complete source code and experimental scripts are available at: https://github.com/bostankhan6/DeepFedNAS

Figures (6)

• Figure 1: DeepFedNAS Pipeline. This diagram illustrates the three core phases of our framework. First, the Offline Pareto Optimal Subnet Search generates a "Pareto Path Subnet Cache" of high-fitness architectures (e.g., 60 subnets in $\sim$20 minutes) using a principled fitness function $\mathcal{F}(\mathcal{A})$. Second, the Federated Supernet Training leverages this cache for "Pareto Path Guided Subnet Sampling," where clients are assigned highly effective architectures for local training, improving supernet weight quality. Finally, the Post-Training Search directly optimizes against $\mathcal{F}(\mathcal{A})$ to find an optimal subnet for deployment (e.g., in $\sim$20 seconds), completely bypassing the need for a costly predictor pipeline.
• Figure 2: Comparison of architectures discovered by our principled, multi-objective search (Principled Path) versus a uniform random sampling baseline. Our search method consistently discovers a Pareto-optimal frontier of architectures with significantly higher network fitness scores for any given computational budget (GigaMACs). This pre-computed cache of elite subnets forms the basis of our path-guided training, avoiding the suboptimal, low-fitness architectures frequently chosen by the baseline's random sampler.
• Figure 3: Accuracy vs. Parameters (Millions). Pareto frontiers for test accuracy percentage against model parameters (Millions) on (a) CIFAR-100 and (b) CINIC-10. DeepFedNAS subnets are blue squares; SuperFedNAS subnets are red circles. Numerical annotations denote MACs in billions.
• Figure 4: Test Accuracy vs. True Latency (CPU and GPU). The CPU target is an Intel Xeon Silver 4210R CPU (@ 2.40 GHz), and the GPU target is an NVIDIA RTX A5000. Annotations show subnet MACs in billions.
• Figure 5: Analysis of cache size effects: (a) average fitness score and (b) standard deviation of fitness, both vs. cache size ($N=60$ marked).