Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TabKANet: Tabular Data Modeling with Kolmogorov-Arnold Network and Transformer

Weihao Gao, Zheng Gong, Zhuo Deng, Fuju Rong, Chucheng Chen, Lan Ma

TL;DR

The TabKANet model for tabular data modeling, which targets the bottlenecks in learning from numerical content, has demonstrated stable and significantly superior performance compared to Neural Networks across multiple public datasets in binary classification, multi-class classification, and regression tasks.

Abstract

Tabular data is the most common type of data in real-life scenarios. In this study, we propose the TabKANet model for tabular data modeling, which targets the bottlenecks in learning from numerical content. We constructed a Kolmogorov-Arnold Network (KAN) based Numerical Embedding Module and unified numerical and categorical features encoding within a Transformer architecture. TabKANet has demonstrated stable and significantly superior performance compared to Neural Networks (NNs) across multiple public datasets in binary classification, multi-class classification, and regression tasks. Its performance is comparable to or surpasses that of Gradient Boosted Decision Tree models (GBDTs). Our code is publicly available on GitHub: https://github.com/AI-thpremed/TabKANet.

TabKANet: Tabular Data Modeling with Kolmogorov-Arnold Network and Transformer

TL;DR

The TabKANet model for tabular data modeling, which targets the bottlenecks in learning from numerical content, has demonstrated stable and significantly superior performance compared to Neural Networks across multiple public datasets in binary classification, multi-class classification, and regression tasks.

Abstract

Tabular data is the most common type of data in real-life scenarios. In this study, we propose the TabKANet model for tabular data modeling, which targets the bottlenecks in learning from numerical content. We constructed a Kolmogorov-Arnold Network (KAN) based Numerical Embedding Module and unified numerical and categorical features encoding within a Transformer architecture. TabKANet has demonstrated stable and significantly superior performance compared to Neural Networks (NNs) across multiple public datasets in binary classification, multi-class classification, and regression tasks. Its performance is comparable to or surpasses that of Gradient Boosted Decision Tree models (GBDTs). Our code is publicly available on GitHub: https://github.com/AI-thpremed/TabKANet.
Paper Structure (20 sections, 2 equations, 5 figures, 11 tables)

This paper contains 20 sections, 2 equations, 5 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Tabular Data Modelling with Neural Network
  4. Kolmogorov-Arnold Network
  5. The TabKANet
  6. Experiments
  7. Datasets
  8. Baseline Models
  9. Results in Binary Classification
  10. Results in Multi-class Classification
  11. Results in Regression Tasks
  12. The Robustness Test of TabKANet
  13. Ablation experiment
  14. Limitation and Future Works
  15. Conclusions
  16. ...and 5 more sections

Figures (5)

  • Figure 1: The architecture design of TabKANet.
  • Figure 2: Illustration of data flow procedure in TabKANet. Implement dual-stream information extraction and achieve unified dimensional representation under Transformer architecture.
  • Figure 3: Robustness evaluation of TabKANet by introducing random noise for categorical and numerical features separately. a) Noise tests in the ON dataset. b) Noise tests in the BA dataset. c) Noise tests in the BL dataset.
  • Figure 4: Impact of different batch sizes on TabKANet performance.
  • Figure 5: Display the effect of outputting two numerical items in two databases using Batch Normalization and Layer Normalization. a)"Informations_Duration" from the ON dataset. b)"Balance" from the BN dataset.