Improving Efficiency in Federated Learning with Optimized Homomorphic Encryption
Feiran Yang
TL;DR
This work addresses the efficiency bottleneck of homomorphic encryption in privacy-preserving federated learning by introducing selective parameter encryption guided by a sensitivity map and embedding differential privacy into encrypted updates. The main method combines an RLWE-based HE scheme with parameter-level packing and a workflow that partitions updates into encrypted and plaintext components, achieving a reported $3\times$ speedup over state-of-the-art HE-based FL methods while maintaining strong privacy guarantees. The theoretical contributions include proofs of correctness, soundness, and differential privacy for the proposed framework, and the implementation demonstrates practical gains in runtime and memory for large-scale models. The results suggest that FL can be deployed more broadly in latency-sensitive and resource-constrained environments (e.g., healthcare and edge computing) without sacrificing privacy, with future work targeting automated sensitivity mapping and hybrid cryptographic techniques for deeper networks.
Abstract
Federated learning is a method used in machine learning to allow multiple devices to work together on a model without sharing their private data. Each participant keeps their private data on their system and trains a local model and only sends updates to a central server, which combines these updates to improve the overall model. A key enabler of privacy in FL is homomorphic encryption (HE). HE allows computations to be performed directly on encrypted data. While HE offers strong privacy guarantees, it is computationally intensive, leading to significant latency and scalability issues, particularly for large-scale models like BERT. In my research, I aimed to address this inefficiency problem. My research introduces a novel algorithm to address these inefficiencies while maintaining robust privacy guarantees. I integrated several mathematical techniques such as selective parameter encryption, sensitivity maps, and differential privacy noise within my algorithms, which has already improved its efficiency. I have also conducted rigorous mathematical proofs to validate the correctness and robustness of the approach. I implemented this algorithm by coding it in C++, simulating the environment of federated learning on large-scale models, and verified that the efficiency of my algorithm is $3$ times the efficiency of the state-of-the-art method. This research has significant implications for machine learning because its ability to improve efficiency while balancing privacy makes it a practical solution! It would enable federated learning to be used very efficiently and deployed in various resource-constrained environments, as this research provides a novel solution to one of the key challenges in federated learning: the inefficiency of homomorphic encryption, as my new algorithm is able to enhance the scalability and resource efficiency of FL while maintaining robust privacy guarantees.