Privacy-Preserving Federated Learning with Differentially Private Hyperdimensional Computing

TL;DR

This work tackles privacy in continuous federated learning for IoT by marrying Hyperdimensional Computing with Differential Privacy in an explainable AI framework. It introduces FedHDPrivacy, an adaptive noise mechanism that tracks cumulative privacy loss and applies only the necessary noise per round, preserving model accuracy during lifelong learning. Empirical results on real manufacturing data show FedHDPrivacy surpassing standard FL baselines (FedAvg, FedProx, FedNova, FedOpt) by up to 37% in accuracy while maintaining efficiency. The framework enables robust privacy against model inversion and membership inference and points toward future extensions such as multimodal data fusion and edge-enabled deployments.

Abstract

Federated Learning (FL) has become a key method for preserving data privacy in Internet of Things (IoT) environments, as it trains Machine Learning (ML) models locally while transmitting only model updates. Despite this design, FL remains susceptible to threats such as model inversion and membership inference attacks, which can reveal private training data. Differential Privacy (DP) techniques are often introduced to mitigate these risks, but simply injecting DP noise into black-box ML models can compromise accuracy, particularly in dynamic IoT contexts, where continuous, lifelong learning leads to excessive noise accumulation. To address this challenge, we propose Federated HyperDimensional computing with Privacy-preserving (FedHDPrivacy), an eXplainable Artificial Intelligence (XAI) framework that integrates neuro-symbolic computing and DP. Unlike conventional approaches, FedHDPrivacy actively monitors the cumulative noise across learning rounds and adds only the additional noise required to satisfy privacy constraints. In a real-world application for monitoring manufacturing machining processes, FedHDPrivacy maintains high performance while surpassing standard FL frameworks - Federated Averaging (FedAvg), Federated Proximal (FedProx), Federated Normalized Averaging (FedNova), and Federated Optimization (FedOpt) - by up to 37%. Looking ahead, FedHDPrivacy offers a promising avenue for further enhancements, such as incorporating multimodal data fusion.

Figures (14)

• Figure 1: Federated learning structure with potential attacks. A federated learning setup where artificial intelligence and machine learning models are trained locally, with parameters aggregated on a central server. Potential attacks include eavesdropping, malicious participants, untrusted servers, and server breaches, all of which may expose local or global models. These attacks exploit model inversion and membership inference techniques to extract sensitive information about the training samples.
• Figure 2: Secure federated learning framework with differential privacy and accuracy trade-off. A federated learning framework enhanced with differential privacy noise added to clients' models. This mechanism helps prevent adversaries from reconstructing training samples or distinguishing original data from random data, thereby safeguarding client information. However, the cumulative noise added over multiple training rounds introduces an accuracy trade-off, gradually impacting model performance.
• Figure 3: Overview of the hyperdimensional computing framework. This framework illustrates the hyperdimensional computing process, covering encoding, training, inference, and retraining phases. In encoding, raw data are transformed into hypervectors. During training, hypervectors from the same class are aggregated to create class hypervectors. Inference involves comparing query hypervectors to class hypervectors for similarity, while retraining adjusts class hypervectors in response to misclassifications, improving model accuracy.
• Figure 4: Experimental setup for federated learning in smart manufacturing. The federated learning structure applied to a smart manufacturing environment with 8 clients, each operating independently to perform quality control tasks. Each client collects local data, processes it, and updates local models, which are then aggregated into a global model to enhance accuracy and maintain data privacy across the network.
• Figure 5: Distance and similarity analysis in hyperdimensional encoding. A visualization of distance and similarity relationships in client data. The main diagonal shows the similarity of encoded samples as a function of distance within each client. The lower triangle displays the similarity distribution between clients, while the upper triangle shows the distance distribution between raw data samples across clients. Together, these elements reveal patterns in data structure, client diversity, and cross-client similarity in the system.

Theorems & Definitions (16)

• Definition 1
• Definition 2
• Definition 3
• Theorem 1
• Theorem 2
• Proof 1
• Theorem 3
• Proof 2
• Lemma 1
• Proof 3