asKAN: Active Subspace embedded Kolmogorov-Arnold Network

TL;DR

This paper tackles the inflexibility of Kolmogorov-Arnold networks (KAN) in representing ridge functions by introducing asKAN, a hierarchical framework that embeds active-subspace projections between KAN levels. The active-subspace step identifies primary ridge directions from gradients and projects inputs onto these directions before passing them to the next KAN, enabling more efficient representations without increasing neuron count. Empirical results show substantial gains in accuracy for function fitting, Poisson equation solving, and sound-field reconstruction compared with standard KAN. The approach offers a practical, scalable method to enhance ridge-function modeling in physics-informed learning tasks with small networks.

Abstract

The Kolmogorov-Arnold Network (KAN) has emerged as a promising neural network architecture for small-scale AI+Science applications. However, it suffers from inflexibility in modeling ridge functions, which is widely used in representing the relationships in physical systems. This study investigates this inflexibility through the lens of the Kolmogorov-Arnold theorem, which starts the representation of multivariate functions from constructing the univariate components rather than combining the independent variables. Our analysis reveals that incorporating linear combinations of independent variables can substantially simplify the network architecture in representing the ridge functions. Inspired by this finding, we propose active subspace embedded KAN (asKAN), a hierarchical framework that synergizes KAN's function representation with active subspace methodology. The architecture strategically embeds active subspace detection between KANs, where the active subspace method is used to identify the primary ridge directions and the independent variables are adaptively projected onto these critical dimensions. The proposed asKAN is implemented in an iterative way without increasing the number of neurons in the original KAN. The proposed method is validated through function fitting, solving the Poisson equation, and reconstructing sound field. Compared with KAN, asKAN significantly reduces the error using the same network architecture. The results suggest that asKAN enhances the capability of KAN in fitting and solving equations in the form of ridge functions.

Figures (9)

• Figure 1: network architecture of KAN and comparison of function values. (a) KAN network architecture; (b) Accurate function values; (c) KAN network fitting results;
• Figure 2: (a) Schematic and (b) flowchart of Active Subspace embedded Kolmogorov-Arnold Network (asKAN)
• Figure 3: The fitting errors and the obtained networks for the ridge function (Eq. \ref{['eq12']}) are presented for the three-level asKAN algorithm.
• Figure 4: The predicted solution and absolute errors for Poisson equation by using KAN and asKAN. (a) and (b) are the predicted solutions of KAN and asKAN, respectively. (c) and (d) are the absolute errors of KAN and asKAN, respectively.
• Figure 5: Changes in loss of the KAN and asKAN with epoch for solving Poisson equation.