CryptoTrain: Fast Secure Training on Encrypted Dataset

TL;DR

Coordinated polynomial convolution is introduced, which encodes only related input values into polynomials, thus drastically reducing the number of computations and overheads in secure training, and achieves a ~5.3× training time reduction compared to prior methods.

Abstract

Secure training, while protecting the confidentiality of both data and model weights, typically incurs significant training overhead. Traditional Fully Homomorphic Encryption (FHE)-based non-inter-active training models are heavily burdened by computationally demanding bootstrapping. To develop an efficient secure training system, we established a foundational framework, CryptoTrain-B, utilizing a hybrid cryptographic protocol that merges FHE with Oblivious Transfer (OT) for handling linear and non-linear operations, respectively. This integration eliminates the need for costly bootstrapping. Although CryptoTrain-B sets a new baseline in performance, reducing its training overhead remains essential. We found that ciphertext-ciphertext multiplication (CCMul) is a critical bottleneck in operations involving encrypted inputs and models. Our solution, the CCMul-Precompute technique, involves precomputing CCMul offline and resorting to the less resource-intensive ciphertext-plaintext multiplication (CPMul) during private training. Furthermore, conventional polynomial convolution in FHE systems tends to encode irrelevant and redundant values into polynomial slots, necessitating additional polynomials and ciphertexts for input representation and leading to extra multiplications. Addressing this, we introduce correlated polynomial convolution, which encodes only related input values into polynomials, thus drastically reducing the number of computations and overheads. By integrating CCMul-Precompute and correlated polynomial convolution into CryptoTrain-B, we facilitate a rapid and efficient secure training framework, CryptoTrain. Extensive experiments demonstrate that CryptoTrain achieves a ~5.3X training time reduction compared to prior methods.

Figures (8)

• Figure 1: Overview of secure training based on cryptographic protocols, e.g., FHE and OT. The lock represents that data and model are encrypted.
• Figure 2: Forward and backward propagations.
• Figure 3: Motivation to precompute CCMul and use CPMul during online training. (a) CCMul is more expensive than CPMul. (b) CCMul needs expensive relinearization. (c) We replace a CCMul with 2 CPMul and negligible CCAdd and PPMul.
• Figure 4: Linear protocol of CryptoTrain-B .
• Figure 5: Non-linear protocols of CryptoTrain.