An End-to-End Homomorphically Encrypted Neural Network
Marcos Florencio, Luiz Alencar, Bianca Lima
TL;DR
This work addresses privacy concerns in neural networks by proposing an end-to-end homomorphically encrypted neural network (HNN) based on CKKS/HEANN, augmented with a Differentiable Soft Argmax layer to calibrate logits in the encrypted domain. The model is trained via backpropagation, maintaining a noise budget $\nu$ to ensure security while controlling accuracy loss, and demonstrates the approach on SST-2 sentiment analysis with a DistilBERT baseline, achieving up to 82.5% of plaintext accuracy under full privacy. Key contributions include the first end-to-end HENN trained with backpropagation, the introduction of the Differentiable Soft Argmax layer, and a practical evaluation showing feasible privacy-preserving inference for NLP. The findings highlight a viable path toward privacy-preserving NLP deployments and motivate further work on interpretability and secure multi-party computation.
Abstract
Every commercially available, state-of-the-art neural network consume plain input data, which is a well-known privacy concern. We propose a new architecture based on homomorphic encryption, which allows the neural network to operate on encrypted data. We show that Homomorphic Neural Networks (HNN) can achieve full privacy and security while maintaining levels of accuracy comparable to plain neural networks. We also introduce a new layer, the Differentiable Soft-Argmax, which allows the calibration of output logits in the encrypted domain, raising the entropy of the activation parameters, thus improving the security of the model, while keeping the overall noise below the acceptable noise budget. Experiments were conducted using the Stanford Sentiment Treebank (SST-2) corpora on the DistilBERT base uncased finetuned SST-2 English sentiment analysis model, and the results show that the HNN model can achieve up to 82.5% of the accuracy of the plain model while maintaining full privacy and security.