HyperFedNet: Communication-Efficient Personalized Federated Learning Via Hypernetwork

TL;DR

HyperFedNet introduces a personalized FL framework that uses a compact hypernetwork to generate the main network's basic-layer parameters conditioned on per-user embedding vectors, while preserving a local personalization layer. The global objective remains $w^{*}= abla_w \left[ \frac{1}{K} \sum_{k=1}^{K} f_k(w_k) \right]$ with $w_k=[w_ heta,w_eta]_k$ and $w_ heta=h(v_k;\varphi)$, enabling reduced communication by exchanging only the hypernetwork parameters $\varphi$. Empirical results on MNIST, FMNIST, CIFAR-10, and CIFAR-100 show that HFN achieves competitive or superior accuracy with substantially lower communication and improved resistance to gradient-based privacy attacks, particularly under non-IID data. The approach is compatible with existing FL algorithms and offers a scalable path to privacy-preserving, communication-efficient, and highly personalized federated learning in heterogeneous environments.

Abstract

In response to the challenges posed by non-independent and identically distributed (non-IID) data and the escalating threat of privacy attacks in Federated Learning (FL), we introduce HyperFedNet (HFN), a novel architecture that incorporates hypernetworks to revolutionize parameter aggregation and transmission in FL. Traditional FL approaches, characterized by the transmission of extensive parameters, not only incur significant communication overhead but also present vulnerabilities to privacy breaches through gradient analysis. HFN addresses these issues by transmitting a concise set of hypernetwork parameters, thereby reducing communication costs and enhancing privacy protection. Upon deployment, the HFN algorithm enables the dynamic generation of parameters for the basic layer of the FL main network, utilizing local database features quantified by embedding vectors as input. Through extensive experimentation, HFN demonstrates superior performance in reducing communication overhead and improving model accuracy compared to conventional FL methods. By integrating the HFN algorithm into the FL framework, HFN offers a solution to the challenges of non-IID data and privacy threats.

Figures (7)

• Figure 1: Hypernetworks diagram
• Figure 2: HFN Framework: Small red networks are hypernetwork and large colored ones are main networks. During parameter aggregation, only the hypernetwork parameters are aggregated.
• Figure 3: Hypernetwork Model Structure.
• Figure 4: Data distribution of $Dir(0.5)$.
• Figure 5: Similarity of main network parameters generated by hypernetwork for different users.