Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GINN-KAN: Interpretability pipelining with applications in Physics Informed Neural Networks

Nisal Ranasinghe, Yu Xia, Sachith Seneviratne, Saman Halgamuge

TL;DR

The paper tackles interpretability in neural networks for scientific modeling by proposing interpretability pipelining and introducing GINN-KAN, a hybrid of Growing Interpretable Neural Networks (GINN) and Kolmogorov Arnold Networks (KAN) trained via backpropagation. It provides end-to-end differentiable architecture details, including $p_i = x_1^{w_{i1}} x_2^{w_{i2}} \cdots x_n^{w_{in}}$ and $y = \sum_{i=1}^{n} a_i p_i$, and a mapping of KAN activations to symbolic functions, enabling equation-level interpretability. Empirical results show GINN-KAN outperforms its constituent models on the Feynman symbolic regression benchmark, especially for non-$LP$ equations with multiplications, and enhances Physics-Informed Neural Networks (PINNs) when used as a drop-in interpretable module across 15 PDEs. Nevertheless, limitations include a restriction to positive inputs and a need for improved regularization, with future work focusing on robustness and broader applicability in scientific DL pipelines.

Abstract

Neural networks are powerful function approximators, yet their ``black-box" nature often renders them opaque and difficult to interpret. While many post-hoc explanation methods exist, they typically fail to capture the underlying reasoning processes of the networks. A truly interpretable neural network would be trained similarly to conventional models using techniques such as backpropagation, but additionally provide insights into the learned input-output relationships. In this work, we introduce the concept of interpretability pipelineing, to incorporate multiple interpretability techniques to outperform each individual technique. To this end, we first evaluate several architectures that promise such interpretability, with a particular focus on two recent models selected for their potential to incorporate interpretability into standard neural network architectures while still leveraging backpropagation: the Growing Interpretable Neural Network (GINN) and Kolmogorov Arnold Networks (KAN). We analyze the limitations and strengths of each and introduce a novel interpretable neural network GINN-KAN that synthesizes the advantages of both models. When tested on the Feynman symbolic regression benchmark datasets, GINN-KAN outperforms both GINN and KAN. To highlight the capabilities and the generalizability of this approach, we position GINN-KAN as an alternative to conventional black-box networks in Physics-Informed Neural Networks (PINNs). We expect this to have far-reaching implications in the application of deep learning pipelines in the natural sciences. Our experiments with this interpretable PINN on 15 different partial differential equations demonstrate that GINN-KAN augmented PINNs outperform PINNs with black-box networks in solving differential equations and surpass the capabilities of both GINN and KAN.

GINN-KAN: Interpretability pipelining with applications in Physics Informed Neural Networks

TL;DR

The paper tackles interpretability in neural networks for scientific modeling by proposing interpretability pipelining and introducing GINN-KAN, a hybrid of Growing Interpretable Neural Networks (GINN) and Kolmogorov Arnold Networks (KAN) trained via backpropagation. It provides end-to-end differentiable architecture details, including and , and a mapping of KAN activations to symbolic functions, enabling equation-level interpretability. Empirical results show GINN-KAN outperforms its constituent models on the Feynman symbolic regression benchmark, especially for non- equations with multiplications, and enhances Physics-Informed Neural Networks (PINNs) when used as a drop-in interpretable module across 15 PDEs. Nevertheless, limitations include a restriction to positive inputs and a need for improved regularization, with future work focusing on robustness and broader applicability in scientific DL pipelines.

Abstract

Neural networks are powerful function approximators, yet their ``black-box" nature often renders them opaque and difficult to interpret. While many post-hoc explanation methods exist, they typically fail to capture the underlying reasoning processes of the networks. A truly interpretable neural network would be trained similarly to conventional models using techniques such as backpropagation, but additionally provide insights into the learned input-output relationships. In this work, we introduce the concept of interpretability pipelineing, to incorporate multiple interpretability techniques to outperform each individual technique. To this end, we first evaluate several architectures that promise such interpretability, with a particular focus on two recent models selected for their potential to incorporate interpretability into standard neural network architectures while still leveraging backpropagation: the Growing Interpretable Neural Network (GINN) and Kolmogorov Arnold Networks (KAN). We analyze the limitations and strengths of each and introduce a novel interpretable neural network GINN-KAN that synthesizes the advantages of both models. When tested on the Feynman symbolic regression benchmark datasets, GINN-KAN outperforms both GINN and KAN. To highlight the capabilities and the generalizability of this approach, we position GINN-KAN as an alternative to conventional black-box networks in Physics-Informed Neural Networks (PINNs). We expect this to have far-reaching implications in the application of deep learning pipelines in the natural sciences. Our experiments with this interpretable PINN on 15 different partial differential equations demonstrate that GINN-KAN augmented PINNs outperform PINNs with black-box networks in solving differential equations and surpass the capabilities of both GINN and KAN.
Paper Structure (15 sections, 3 equations, 3 figures, 4 tables)

This paper contains 15 sections, 3 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Neural Network Interpretability
  3. Limitations of GINN and KAN
  4. Physics-informed Neural Networks
  5. Methodology
  6. GINN-KAN
  7. GINN-KAN Augmented PINNs
  8. Experiment Setup
  9. GINN-KAN
  10. GINN-KAN Augmented PINNs
  11. Results and Discussion
  12. GINN-KAN
  13. GINN-KAN Augmented PINNs
  14. Conclusion
  15. Appendix

Figures (3)

  • Figure 1: The architecture of GINN-KAN. It consists of two parallel GINN blocks, followed by a KAN block. Each GINN is composed of multiple parallel power-term approximation (PTA) blocks, with each block aiming to discover a single multiplication term in the equation. The entire network is trained end-to-end using backpropagation, which trains the KAN and the PTA blocks within the GINNs. The example shows how GINN-KAN could discover the equation $y = sin(x_1x_2) + e^{x_1}$
  • Figure 2: The architecture of GINN-KAN augmented PINNs. The proposed GINN-KAN module replaces the fully connected networks in the conventional PINNs, converting the black-box PINN into an interpretable PINN, without making any changes to the training strategy, since this interpretable PINN can still be trained using backpropagation
  • Figure 3: Performance of GINN, KAN and GINN-KAN when compared to existing symbolic regression methods. GINN, KAN and GINN-KAN are the only methods that can be trained using backpropagation. Experiments are performed on 114 SR datasets from the Feynman symbolic regression benchmark. The percentage of datasets with $R^2 > 0.99$ is reported. The bold labels show the methods that can be trained using backpropagation.