Data-driven model discovery with Kolmogorov-Arnold networks

TL;DR

The KAN framework with a simple structure is capable of accurately capturing the complex behavior of dynamical systems that do not meet the sparsity requirement while offering greater interpretability compared to conventional neural networks, which provides insights into the dynamics generating the data.

Abstract

Data-driven model discovery of complex dynamical systems is typically done using sparse optimization, but it has a fundamental limitation: sparsity in that the underlying governing equations of the system contain only a small number of elementary mathematical terms. Examples where sparse optimization fails abound, such as the classic Ikeda or optical-cavity map in nonlinear dynamics and a large variety of ecosystems. Exploiting the recently articulated Kolmogorov-Arnold networks, we develop a general model-discovery framework for any dynamical systems including those that do not satisfy the sparsity condition. In particular, we demonstrate non-uniqueness in that a large number of approximate models of the system can be found which generate the same invariant set with the correct statistics such as the Lyapunov exponents and Kullback-Leibler divergence. An analogy to shadowing of numerical trajectories in chaotic systems is pointed out.

Figures (4)

• Figure 1: Basics of KAN. (a) Kolmogorov-Arnold theorem and neural network. (b) Schematic illustration of two different structures (blue and green) leading to two different functions $\mathbf{M}(\mathbf{x})$ and $\mathbf{L}(\mathbf{x})$ that generate the same dynamics as $\mathbf{x}_{n+1} = \mathbf{F}(\mathbf{x}_n)$ in the relevant phase-space domain.
• Figure 2: KANs applied to the Ikeda map. (a) A KAN structure with 2 input, 4 hidden, and 2 output nodes. (b) Training (red) and testing (black dashed) loss curves. (c) Chaotic attractor during the training phase (blue - ground truth; orange - KAN produced). (d,e) Time series during the training. The blue and orange traces overlap well, signifying a high training accuracy. (f) Chaotic attractor during testing (blue - ground truth; orange - KAN produced). (g,h) The corresponding time series. While the predicted time series diverges from the ground truth after a few iterations due to chaos, the KAN generates the correct attractor in the pertinent phase-space domain. The true Lyapunov exponents of the chaotic attractor are $[0.5025,-0.7263]$. The KAN predicted model gives the values of the two exponents as $[0.5075,-0.7182]$, agreeing with the ground truth.
• Figure 3: A KAN configuration generating a different representation of the Ikeda map but with the same chaotic attractor. The KAN has 2 input, 10 hidden, and 2 output nodes. Legends are the same as those in Fig. \ref{['fig:Ikeda1']}. The two Lyapunov exponents of the KAN predicted model are $[0.5033,-0.7311]$, which again agrees with the true exponents.
• Figure 4: KAN applied to a chaotic ecosystem. (a) KAN structure with 3 input and 3 output nodes. (b) Training and testing loss curves. (c) KAN generated attractor during the training phase (orange), which agrees completely with the ground truth (blue). (d-f) KAN generated time series (orange) in agreement with the true time series (blue). (g-j) Similar to (c-f) but for the testing phase. Due to chaos, the KAN generated time series diverges from the true ones from the same initial condition, but the KAN attractor agrees with the true one. The true Lyapunov exponents are $[0.0053, 0, -0.2288]$. The exponents of the KAN-generated attractor are consistent: $[0.0095, -5.8\times 10^{-6}, -0.3932]$. The errors arise from the implicit numerical evaluation of the Jacobian matrix.