Shift-Invariant Attribute Scoring for Kolmogorov-Arnold Networks via Shapley Value

TL;DR

The paper tackles interpretability and pruning in Kolmogorov–Arnold Networks (KANs), where standard magnitude-based pruning is unreliable under covariate shift. It introduces ShapKAN, a Shapley-value–based pruning framework that treats layer neurons as cooperative-game players and uses permutation sampling with antithetic variance reduction to efficiently approximate neuron attributions, applied in a bottom-up, multi-layer manner. Across synthetic and real-world datasets, ShapKAN yields shift-invariant attributions, preserves ground-truth symbolic structure under covariate shift, and enables more effective compression while maintaining generalization. This approach enhances interpretability and enables deployment of KANs in resource-constrained settings, while remaining compatible with advanced KAN variants like MultKAN 2.0.

Abstract

For many real-world applications, understanding feature-outcome relationships is as crucial as achieving high predictive accuracy. While traditional neural networks excel at prediction, their black-box nature obscures underlying functional relationships. Kolmogorov--Arnold Networks (KANs) address this by employing learnable spline-based activation functions on edges, enabling recovery of symbolic representations while maintaining competitive performance. However, KAN's architecture presents unique challenges for network pruning. Conventional magnitude-based methods become unreliable due to sensitivity to input coordinate shifts. We propose \textbf{ShapKAN}, a pruning framework using Shapley value attribution to assess node importance in a shift-invariant manner. Unlike magnitude-based approaches, ShapKAN quantifies each node's actual contribution, ensuring consistent importance rankings regardless of input parameterization. Extensive experiments on synthetic and real-world datasets demonstrate that ShapKAN preserves true node importance while enabling effective network compression. Our approach improves KAN's interpretability advantages, facilitating deployment in resource-constrained environments.

Figures (5)

• Figure 2: Cross-domain robustness comparison of neuron importance ranking between Vanilla KAN and ShapKAN. Each circle represents a node's importance rank evaluated by scoring methods. Shadow indicates the most substantial nodes. Numeric record of mean, standard deviations and pruning set are reported in the Appendix \ref{['appendix: Neuron Importance Scores Supplement']}.
• Figure 3: Symbolic regression comparison on the multiplication dataset under covariate shift. $R^2$ measures symbolic similarity ($1$ = exact match). ShapKAN recovers $\hat{f}(x_1,x_2) \approx x_1x_2 + c$ (small constant), while Vanilla KAN yields $\hat{f}(x_1,x_2) \approx -0.01x_2 + 0.01e^{x_1} + c$. Red edges denote validated symbolic functions, blue edges represent refitted functions.
• Figure 4: Convergence analysis of permutation sampling methods on simulated datasets. Left: Standard permutation sampling. Right: Antithetical permutation sampling.
• Figure 5: Comparison of generalization capacity. Mean and standard deviation are reported based on 10 times independent experiments. For accuracy and Area under the Receiver Operating Characteristic Curve (AUROC), higher are better; for RMSE, lower is better. In Airbnb dataset, DropKAN's RMSE significantly exceeds 0.3 when the ratio is above 0.6.
• Figure 6: Validation of SV approximation accuracy against ground-truth values across different sampling sizes. As bias decreases, points align more closely with the diagonal line.