Demonstrating the Efficacy of Kolmogorov-Arnold Networks in Vision Tasks

TL;DR

This work evaluates Kolmogorov-Arnold Networks (KAN) for vision by introducing the KAN-Mixer, a patch-based architecture that uses spline-parameterized, learnable activations. The model processes image patches with Per-Patch KAN followed by a Mixer Stack comprising Token Mixing KAN and Channel Mixing KAN, culminating in an Output KAN for classification. Across MNIST, CIFAR-10, and CIFAR-100, the KAN-Mixer achieves strong results on MNIST (≈98.16%) and surpasses the original MLP-Mixer on CIFAR datasets but remains below ResNet-18 on CIFAR-10/100, indicating both promise and room for improvement. The paper highlights KANs as a viable, interpretable alternative for vision tasks with potential gains through further tuning and architectural refinements.

Abstract

In the realm of deep learning, the Kolmogorov-Arnold Network (KAN) has emerged as a potential alternative to multilayer projections (MLPs). However, its applicability to vision tasks has not been extensively validated. In our study, we demonstrated the effectiveness of KAN for vision tasks through multiple trials on the MNIST, CIFAR10, and CIFAR100 datasets, using a training batch size of 32. Our results showed that while KAN outperformed the original MLP-Mixer on CIFAR10 and CIFAR100, it performed slightly worse than the state-of-the-art ResNet-18. These findings suggest that KAN holds significant promise for vision tasks, and further modifications could enhance its performance in future evaluations.Our contributions are threefold: first, we showcase the efficiency of KAN-based algorithms for visual tasks; second, we provide extensive empirical assessments across various vision benchmarks, comparing KAN's performance with MLP-Mixer, CNNs, and Vision Transformers (ViT); and third, we pioneer the use of natural KAN layers in visual tasks, addressing a gap in previous research. This paper lays the foundation for future studies on KANs, highlighting their potential as a reliable alternative for image classification tasks.