Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Dynamic Framework for Grid Adaptation in Kolmogorov-Arnold Networks

Spyros Rigas, Thanasis Papaioannou, Panagiotis Trakadas, Georgios Alexandridis

TL;DR

This work reframes knot allocation in Kolmogorov--Arnol'd Networks as a density estimation problem using Importance Density Functions, extending beyond traditional input-density heuristics. It introduces a curvature-based IDF that concentrates knots in regions of high geometric complexity, implemented via $w^{(s)}_{\text{curv}}$ and the inverse-CDF knot placement $t_m = \hat{F}_d^{-1}(q_m)$. Empirical results across synthetic functions, a Feynman dataset subset, and Helmholtz PDEs show substantial relative error reductions (up to $34\%$ in some cases) and improved training stability, with Wilcoxon tests confirming statistical significance in multiple benchmarks. The approach remains computationally efficient and scalable, offering a principled path toward training-dynamics driven grid refinement in scientific machine learning tasks.

Abstract

Kolmogorov-Arnold Networks (KANs) have recently demonstrated promising potential in scientific machine learning, partly due to their capacity for grid adaptation during training. However, existing adaptation strategies rely solely on input data density, failing to account for the geometric complexity of the target function or metrics calculated during network training. In this work, we propose a generalized framework that treats knot allocation as a density estimation task governed by Importance Density Functions (IDFs), allowing training dynamics to determine grid resolution. We introduce a curvature-based adaptation strategy and evaluate it across synthetic function fitting, regression on a subset of the Feynman dataset and different instances of the Helmholtz PDE, demonstrating that it significantly outperforms the standard input-based baseline. Specifically, our method yields average relative error reductions of 25.3% on synthetic functions, 9.4% on the Feynman dataset, and 23.3% on the PDE benchmark. Statistical significance is confirmed via Wilcoxon signed-rank tests, establishing curvature-based adaptation as a robust and computationally efficient alternative for KAN training.

A Dynamic Framework for Grid Adaptation in Kolmogorov-Arnold Networks

TL;DR

This work reframes knot allocation in Kolmogorov--Arnol'd Networks as a density estimation problem using Importance Density Functions, extending beyond traditional input-density heuristics. It introduces a curvature-based IDF that concentrates knots in regions of high geometric complexity, implemented via and the inverse-CDF knot placement . Empirical results across synthetic functions, a Feynman dataset subset, and Helmholtz PDEs show substantial relative error reductions (up to in some cases) and improved training stability, with Wilcoxon tests confirming statistical significance in multiple benchmarks. The approach remains computationally efficient and scalable, offering a principled path toward training-dynamics driven grid refinement in scientific machine learning tasks.

Abstract

Kolmogorov-Arnold Networks (KANs) have recently demonstrated promising potential in scientific machine learning, partly due to their capacity for grid adaptation during training. However, existing adaptation strategies rely solely on input data density, failing to account for the geometric complexity of the target function or metrics calculated during network training. In this work, we propose a generalized framework that treats knot allocation as a density estimation task governed by Importance Density Functions (IDFs), allowing training dynamics to determine grid resolution. We introduce a curvature-based adaptation strategy and evaluate it across synthetic function fitting, regression on a subset of the Feynman dataset and different instances of the Helmholtz PDE, demonstrating that it significantly outperforms the standard input-based baseline. Specifically, our method yields average relative error reductions of 25.3% on synthetic functions, 9.4% on the Feynman dataset, and 23.3% on the PDE benchmark. Statistical significance is confirmed via Wilcoxon signed-rank tests, establishing curvature-based adaptation as a robust and computationally efficient alternative for KAN training.
Paper Structure (12 sections, 8 equations, 2 figures, 2 tables)

This paper contains 12 sections, 8 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Kolmogorov--Arnold Networks
  3. Proposed Framework
  4. Generalized Knot Allocation
  5. Input-Based Adaptation as a Special Case
  6. Curvature-Based Adaptation
  7. An Illustrative Example
  8. Experimental Results
  9. Benchmark I: Synthetic Functions
  10. Benchmark II: Feynman Dataset
  11. Benchmark III: Helmholtz Equation
  12. Conclusion & Outlook

Figures (2)

  • Figure 1: Visual comparison of grid adaptation strategies on a 1D sharp Gaussian regression task. (Top) The model predictions from the input-based (dashed red) and curvature-based (dashed blue) KAN models. The training data, sampled from a uniform distribution, is shown in light gray. (Middle) The resulting knot allocation for the input-based strategy. (Bottom) The knot allocation for the proposed curvature-based strategy.
  • Figure 2: Comparative evaluation on the Helmholtz PDE benchmark across four increasing frequency configurations. The grouped bar chart showcases the median relative $L^2$ error in logarithmic scale, with error bars indicating the standard deviation over 3 independent runs.