Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Network Training on Encrypted Data with TFHE

Luis Montero, Jordan Frery, Celia Kherfallah, Roman Bredehoft, Andrei Stoian

TL;DR

The paper tackles confidential training of neural networks on encrypted data, addressing leakage risks in multi-party settings where data may be horizontally or vertically distributed. It introduces a unified training pipeline built on Fully Homomorphic Encryption using TFHE, with PBS-enabled arithmetic and a quantized integer representation to support logistic regression and a one-hidden-layer MLP. Key contributions include offline quantization calibration to produce an integer graph with LUT-based activations, a TFHE-based compilation into secure circuits, a bit-removal rounding operator for scaling with $M = 2^{-n_{r}} M_0$, and a mini-batch scheme that refreshes ciphertexts. Experimental results show convergence to plaintext fp32 accuracy on two datasets and competitive latency metrics, demonstrating practical leakage-free training.

Abstract

We present an approach to outsourcing of training neural networks while preserving data confidentiality from malicious parties. We use fully homomorphic encryption to build a unified training approach that works on encrypted data and learns quantized neural network models. The data can be horizontally or vertically split between multiple parties, enabling collaboration on confidential data. We train logistic regression and multi-layer perceptrons on several datasets.

Neural Network Training on Encrypted Data with TFHE

TL;DR

The paper tackles confidential training of neural networks on encrypted data, addressing leakage risks in multi-party settings where data may be horizontally or vertically distributed. It introduces a unified training pipeline built on Fully Homomorphic Encryption using TFHE, with PBS-enabled arithmetic and a quantized integer representation to support logistic regression and a one-hidden-layer MLP. Key contributions include offline quantization calibration to produce an integer graph with LUT-based activations, a TFHE-based compilation into secure circuits, a bit-removal rounding operator for scaling with , and a mini-batch scheme that refreshes ciphertexts. Experimental results show convergence to plaintext fp32 accuracy on two datasets and competitive latency metrics, demonstrating practical leakage-free training.

Abstract

We present an approach to outsourcing of training neural networks while preserving data confidentiality from malicious parties. We use fully homomorphic encryption to build a unified training approach that works on encrypted data and learns quantized neural network models. The data can be horizontally or vertically split between multiple parties, enabling collaboration on confidential data. We train logistic regression and multi-layer perceptrons on several datasets.
Paper Structure (14 sections, 1 equation, 3 figures, 1 table)

This paper contains 14 sections, 1 equation, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Prior work
  3. Encrypted training
  4. Integer arithmetic quantized training
  5. Method
  6. Training computation graph
  7. Quantization
  8. Compilation to TFHE
  9. TFHE Rounding operator
  10. Mini-batch training and weight noise
  11. Experimental Evaluation
  12. Models and Datasets
  13. Results
  14. Conclusion

Figures (3)

  • Figure 1: Training computation graph for Logistic Regression showing a single batch. X contains the training data $X$, Y the labels $y$. Initial trained parameters are stored in bias_0 (bias $b$), weight_0 (weights $w$). MatMul, Add and ReduceSum operations have calibrated input quantizers while Mul, Div, Sigmoid are computed with PBS.
  • Figure 2: Rounded PBS removing the 1st LSB of a 5-bit value: first the least significant bit is shifted to become the most significant bit. Next, a 1-bit PBS moves the bit back to the 5-th position. A final subtraction removes the bit from the original value.
  • Figure 3: Accuracy during training for Logistic Regression and MLP on two datasets