Efficient Privacy-Preserving KAN Inference Using Homomorphic Encryption

TL;DR

The paper tackles privacy-preserving inference for Kolmogorov-Arnold Networks (KANs) using Homomorphic Encryption, addressing the challenge of non-linear SiLU activations and B-spline computations. It introduces a task-specific SiLU polynomial approximation with adaptive range guided by data distribution and a fast B-spline basis computation via Repeat Packing, lazy combination, and comparison-based methods. The proposed HE-KAN inference framework achieves accuracy close to plaintext KANs and outperforms plaintext MLPs on image tasks, while delivering up to about a 7x latency speedup on CIFAR-10 compared with naive HE implementations. These advances enable practical, secure inference for interpretable KANs in resource-constrained settings, with demonstrated effectiveness on both symbolic formulas and image classification tasks.

Abstract

The recently proposed Kolmogorov-Arnold Networks (KANs) offer enhanced interpretability and greater model expressiveness. However, KANs also present challenges related to privacy leakage during inference. Homomorphic encryption (HE) facilitates privacy-preserving inference for deep learning models, enabling resource-limited users to benefit from deep learning services while ensuring data security. Yet, the complex structure of KANs, incorporating nonlinear elements like the SiLU activation function and B-spline functions, renders existing privacy-preserving inference techniques inadequate. To address this issue, we propose an accurate and efficient privacy-preserving inference scheme tailored for KANs. Our approach introduces a task-specific polynomial approximation for the SiLU activation function, dynamically adjusting the approximation range to ensure high accuracy on real-world datasets. Additionally, we develop an efficient method for computing B-spline functions within the HE domain, leveraging techniques such as repeat packing, lazy combination, and comparison functions. We evaluate the effectiveness of our privacy-preserving KAN inference scheme on both symbolic formula evaluation and image classification. The experimental results show that our model achieves accuracy comparable to plaintext KANs across various datasets and outperforms plaintext MLPs. Additionally, on the CIFAR-10 dataset, our inference latency achieves over 7 times speedup compared to the naive method.

Figures (5)

• Figure 1: Obstacles in transitioning from privacy-preserving inference of existing neural networks to privacy-preserving inference of KAN.
• Figure 2: Structure of HE-friendly KAN. The specific structure of the KANLayer is shown within the dashed box.
• Figure 3: Toy example of B-spline Basis Approximation when $g=2$ and $k=1$.
• Figure 4: Comparison of our method with Remez zimerman2024converting and OLS zheng2022keyword for fitting the SiLU activation function on the MNIST dataset.
• Figure 5: Comparison of inference latency between our method and the naive implementation across different datasets