DP-KAN: Differentially Private Kolmogorov-Arnold Networks

TL;DR

The paper tackles enabling differentially private training for Kolmogorov-Arnold Networks (KAN) and benchmarks them against MLPs. It adopts differential privacy via DP-SGD, employing DP-Adam for enhanced optimization, and evaluates on tabular regression tasks and the MNIST dataset with a FasterKAN variant. The key finding is that private KAN performance is comparable to private MLP performance, with similar degradation under privacy constraints; FasterKAN also demonstrates higher parameter efficiency while maintaining accuracy. This work positions KAN as a practical, privacy-preserving alternative for ML tasks, broadening the toolkit for differential privacy in neural architectures.

Abstract

We study the Kolmogorov-Arnold Network (KAN), recently proposed as an alternative to the classical Multilayer Perceptron (MLP), in the application for differentially private model training. Using the DP-SGD algorithm, we demonstrate that KAN can be made private in a straightforward manner and evaluated its performance across several datasets. Our results indicate that the accuracy of KAN is not only comparable with MLP but also experiences similar deterioration due to privacy constraints, making it suitable for differentially private model training.

Figures (1)

• Figure 1: Validation accuracy vs the number of parameters and the sizes of the hidden layers for FasterKAN, MLP, DP MLP, and DP FasterKAN on MNIST with $(0.87, 10^{-5})$ DP.

Theorems & Definitions (1)

• Definition 2.1: $(\varepsilon, \delta)$-differential privacy Dwork_DP_original