Volley Revolver: A Novel Matrix-Encoding Method for Privacy-Preserving Neural Networks (Inference)
John Chiang
TL;DR
This work addresses privacy-preserving neural network inference by outsourcing encrypted data to a cloud while preserving data confidentiality through a CKKS-based homomorphic encryption framework. It introduces Volley Revolver, a novel matrix-encoding scheme that encodes $A$ directly and $B$ (via a transposed, padded form) as $\bar{B}$ so that the matrix product $C = A \cdot B$ can be computed efficiently under HE, complemented by SumForConv for partial convolution sums and a virtual ciphertext construction to enable parallel processing of multiple images. The approach is demonstrated by a CNN for MNIST, achieving $98.61\%$ accuracy on 10{,}000 test images while processing 32 images per ciphertext, with a runtime of about $287$ seconds on a 40 vCPU cloud; the data owner uploads a single ciphertext (~$19.8$ MB) and the model provider transmits ~52 ciphertexts (~1 GB). The contributions lay groundwork for practical private inference and hint at extending the encoding approach to training, with potential applicability to color and high-resolution data in real-world deployments.
Abstract
In this work, we present a novel matrix-encoding method that is particularly convenient for neural networks to make predictions in a privacy-preserving manner using homomorphic encryption. Based on this encoding method, we implement a convolutional neural network for handwritten image classification over encryption. For two matrices $A$ and $B$ to perform homomorphic multiplication, the main idea behind it, in a simple version, is to encrypt matrix $A$ and the transpose of matrix $B$ into two ciphertexts respectively. With additional operations, the homomorphic matrix multiplication can be calculated over encrypted matrices efficiently. For the convolution operation, we in advance span each convolution kernel to a matrix space of the same size as the input image so as to generate several ciphertexts, each of which is later used together with the ciphertext encrypting input images for calculating some of the final convolution results. We accumulate all these intermediate results and thus complete the convolution operation. In a public cloud with 40 vCPUs, our convolutional neural network implementation on the MNIST testing dataset takes $\sim$ 287 seconds to compute ten likelihoods of 32 encrypted images of size $28 \times 28$ simultaneously. The data owner only needs to upload one ciphertext ($\sim 19.8$ MB) encrypting these 32 images to the public cloud.