Speed-up of Data Analysis with Kernel Trick in Encrypted Domain

TL;DR

This work tackles the performance bottleneck of high-dimensional data analysis under homomorphic encryption by introducing a kernel-trick optimizer that is agnostic to the underlying HE scheme. By precomputing and reusing kernel elements, the approach dramatically reduces heavy multiplications and yields near-constant time with respect to data dimension, improving both ML and STAT tasks in the encrypted domain. The authors provide a comprehensive complexity analysis across arithmetic and Boolean HE, demonstrate substantial empirical speedups on SVM, PCA, k-means, and k-NN, and show that the kernel method enhances training prospects in secure settings. The result is a practical, user-friendly optimization that can synergize with existing HE accelerators to enable scalable privacy-preserving learning and analysis.

Abstract

Homomorphic encryption (HE) is pivotal for secure computation on encrypted data, crucial in privacy-preserving data analysis. However, efficiently processing high-dimensional data in HE, especially for machine learning and statistical (ML/STAT) algorithms, poses a challenge. In this paper, we present an effective acceleration method using the kernel method for HE schemes, enhancing time performance in ML/STAT algorithms within encrypted domains. This technique, independent of underlying HE mechanisms and complementing existing optimizations, notably reduces costly HE multiplications, offering near constant time complexity relative to data dimension. Aimed at accessibility, this method is tailored for data scientists and developers with limited cryptography background, facilitating advanced data analysis in secure environments.

Figures (9)

• Figure 1: (P1) Parallel computation structure for evaluating kernel elements. Expensive HE multiplications can be pre-computed in parallel during the initial stage for enhanced performance. Each kernel element evaluation can be assigned to a separate processor, enabling concurrent computation over the kernel matrix $\mathbf{K}$.
• Figure 2: (P2) Dimensionless---constant time complexity w.r.t. dimension. While general ML/STAT algorithms require the computation of inner products involving costly multiplications, the kernel method bypasses these dot products by utilizing kernel elements or scalars.
• Figure 3: Example of the $k-$means algorithm. The kernel method lessens the multiplication number by approximately a factor of $d$ (P1). The inner products or the kernel element is computed parallel from the initial stage (P2). The total time complexity is nearly constant with respect to dimension $d$.
• Figure 4: $k$-means simulation: This figure demonstrates the effectiveness of the kernel method in $k$-means across different domains—plain, TFHE, CKKS, and B/FV. The parameters were fixed at $k=3$ and $t=10$, and the log ratio of general time ($t_{gen}$) to kernel time ($t_{ker}$) was compared.
• Figure 5: Execution Time Comparison Between ML's General and Kernel Methods Showing Kernel's Dimensionless Property (P2).