Decentralized Privacy-Preserving Federal Learning of Computer Vision Models on Edge Devices
Damian Harenčák, Lukáš Gajdošech, Martin Madaras
TL;DR
The paper addresses privacy risks in federated learning for edge-based 3D computer vision, noting that gradient updates can leak private information. It analyzes server-side privacy methods (homomorphic encryption, including Paillier and CKKS) and client-side privacy strategies (gradient compression and gradient noising), and discusses alternative FL topologies (split learning and swarm learning) as potential remedies, validated via a proof-of-concept on NVIDIA Jetson TX2. Key findings show Paillier is impractical for large models, while CKKS provides viable secure aggregation with manageable overhead; gradient noising offers a strong privacy-utility balance, whereas gradient compression improves reconstruction resistance but can degrade accuracy at high prune levels; segmentation reconstruction remains difficult for realistic data. The work demonstrates practical, lightweight privacy-preserving FL options for edge devices and outlines open problems such as meaningful privacy metrics, and the exploration of lighter, certifiable architectures for robust 3D vision deployments.
Abstract
Collaborative training of a machine learning model comes with a risk of sharing sensitive or private data. Federated learning offers a way of collectively training a single global model without the need to share client data, by sharing only the updated parameters from each client's local model. A central server is then used to aggregate parameters from all clients and redistribute the aggregated model back to the clients. Recent findings have shown that even in this scenario, private data can be reconstructed only using information about model parameters. Current efforts to mitigate this are mainly focused on reducing privacy risks on the server side, assuming that other clients will not act maliciously. In this work, we analyzed various methods for improving the privacy of client data concerning both the server and other clients for neural networks. Some of these methods include homomorphic encryption, gradient compression, gradient noising, and discussion on possible usage of modified federated learning systems such as split learning, swarm learning or fully encrypted models. We have analyzed the negative effects of gradient compression and gradient noising on the accuracy of convolutional neural networks used for classification. We have shown the difficulty of data reconstruction in the case of segmentation networks. We have also implemented a proof of concept on the NVIDIA Jetson TX2 module used in edge devices and simulated a federated learning process.