Probabilistic Kolmogorov-Arnold Network

TL;DR

The present paper proposes a method for estimating probability distributions of the outputs in the case of aleatoric uncertainty, which is applicable to any regression model and combines it with KANs, since the specific structure of KANs leads to computationally-efficient models' construction.

Abstract

The Kolmogorov-Arnold network (KAN) is a regression model that is based on a representation of an arbitrary continuous multivariate function by a composition of functions of a single variable. Experimentally-obtained datasets for regression models typically include uncertainties, which in some cases, cannot be neglected. The conventional way to account for the latter is to model confidence intervals of the systems' outputs in addition to the expected values of the outputs. However, such information may be insufficient, and in some cases, researchers aim to obtain probability distributions of the outputs. The present paper proposes a method for estimating probability distributions of the outputs in the case of aleatoric uncertainty (i.e. for systems that produce different outputs each time an experiment is executed with the same inputs). The suggested approach covers input-dependent probability distributions of the outputs and is capable of capturing the multi-modality, as well as the variation of the distribution type with the inputs. Although the method is applicable to any regression model, the present paper combines it with KANs, since the specific structure of KANs leads to computationally-efficient models' construction. The source code is available online.

Figures (4)

• Figure 1: A schematic illustration of the concept of the proposed algorithm. The ensemble of models is built for obtaining the probabilistic properties of the output.
• Figure 2: Probability density of the output for the dice example: the reference solution obtained using the Monte Carlo (MC) sampling (solid line) and the estimation made using the divisive data re-sorting (DDR) algorithm (bar chart).
• Figure 3: The empirical cumulative distribution functions (ECDFs) of the output of the considered stochastic system obtained using the Monte Carlo (MC) sampling (black) and using the realisations of the divisive data re-sorting (DDR) algorithm (grey). Averages over the realisations are shown in red. The corresponding probability density functions (PDFs) obtained using the MC sampling are shown in the insets. Subfigures (a)-(d) correspond to inputs $X^1$-$X^4$.
• Figure 4: The mean and the standard deviation for $100$ input points. The points are sorted based on their $X_1$ value. The values are obtained using the MC sampling (red squares), using the DDR ensembles (blue crosses), and using the ensembles of models trained on randomly selected disjoint sets of records (grey pluses).