Energy Efficient Federated Learning with Hyperdimensional Computing over Wireless Communication Networks

TL;DR

A novel FL with hyperdimensional computing and differential privacy (FL-HDC-DP) framework is proposed, which achieves up to 83.3% total energy reduction compared with the baseline, while attaining about 90% accuracy in approximately 3.5X fewer communication rounds than the NN baseline.

Abstract

In this paper, we investigate a problem of minimizing total energy consumption for secure federated learning (FL) over wireless edge networks. To address the high computational cost and privacy challenges in conventional FL with neural networks (NN) for resource-constrained users, we propose a novel FL with hyperdimensional computing and differential privacy (FL-HDC-DP) framework. In the considered model, each edge user employs hyperdimensional computing (HDC) for local training, which replaces complex neural updates with simple hypervector operations, and applies differential privacy (DP) noise to protect transmitted model information. We optimize the total energy of computation and communication under both latency and privacy constraints. We formulate the problem as an optimization that minimizes the total energy of all users by jointly allocating HDC dimension, transmission time, system bandwidth, transmit power, and CPU frequency. To solve this problem, a sigmoid-variant function is proposed to characterize the relationship between the HDC dimension and the convergence rounds required to reach a target accuracy. Based on this model, we develop two alternating optimization algorithms, where closed-form expressions for time, frequency, bandwidth, and power allocations are derived at each iteration. Since the iterative algorithm requires a feasible initialization, we construct a feasibility problem and obtain feasible initial resource parameters by solving a per round transmission time minimization problem. Simulation results demonstrate that the proposed FL-HDC-DP framework achieves up to 83.3% total energy reduction compared with the baseline, while attaining about 90% accuracy in approximately 3.5X fewer communication rounds than the NN baseline.

Figures (9)

• Figure 1: Overview of HDC model encoding, training, inference, and retraining.
• Figure 2: Illustration of the considered model for FL-HDC over wireless communication networks.
• Figure 3: Accuracy vs. Epochs of FL-HDC under IID data with different hypervector dimensions: (a) without DP noise and (b) with DP noise ($\epsilon = 20$).
• Figure 4: Accuracy vs. epochs of FL-HDC-DP and FL-NN-DP under different data distributions and privacy budgets: (a) IID and non-IID data with $\epsilon = 20$ and (b) non-IID data with $\epsilon = 10, 20$.
• Figure 5: Fitting performance of the proposed sigmoid-variant model for the relationship between the hypervector dimension $d$ and the number of rounds to converge $J_d$ under different target accuracies: (a) without DP noise and (b) with DP noise ($\epsilon = 20$).

Theorems & Definitions (11)

• Lemma 1: Per-round Gaussian noise to ensure $(\varepsilon,\delta)$-DP after $J$ rounds
• Theorem 1
• proof
• Proposition 1
• Theorem 2
• Proposition 2
• proof
• Lemma 2
• proof
• Proposition 3