Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Vision KAN: Towards an Attention-Free Backbone for Vision with Kolmogorov-Arnold Networks

Zhuoqin Yang, Jiansong Zhang, Xiaoling Luo, Xu Wu, Zheng Lu, Linlin Shen

TL;DR

This work tackles the high computational cost and limited interpretability of attention in vision backbones by proposing Vision KAN (ViK), an attention-free architecture built upon Kolmogorov-Arnold networks. ViK replaces self-attention with a unified MultiPatch-RBFKAN token mixer that fuses patch-wise nonlinear RBF modeling, axis-wise separable mixing, and a low-rank global path to enable scalable long-range interactions. The approach achieves competitive ImageNet-1K accuracy with linear complexity, validated through extensive ablations and interpretability analyses of learned RBF mappings. Overall, ViK demonstrates that principled function-based token mixing can match or exceed attention-based performance while dramatically reducing computational overhead, offering a viable direction for efficient vision backbones.

Abstract

Attention mechanisms have become a key module in modern vision backbones due to their ability to model long-range dependencies. However, their quadratic complexity in sequence length and the difficulty of interpreting attention weights limit both scalability and clarity. Recent attention-free architectures demonstrate that strong performance can be achieved without pairwise attention, motivating the search for alternatives. In this work, we introduce Vision KAN (ViK), an attention-free backbone inspired by the Kolmogorov-Arnold Networks. At its core lies MultiPatch-RBFKAN, a unified token mixer that combines (a) patch-wise nonlinear transform with Radial Basis Function-based KANs, (b) axis-wise separable mixing for efficient local propagation, and (c) low-rank global mapping for long-range interaction. Employing as a drop-in replacement for attention modules, this formulation tackles the prohibitive cost of full KANs on high-resolution features by adopting a patch-wise grouping strategy with lightweight operators to restore cross-patch dependencies. Experiments on ImageNet-1K show that ViK achieves competitive accuracy with linear complexity, demonstrating the potential of KAN-based token mixing as an efficient and theoretically grounded alternative to attention.

Vision KAN: Towards an Attention-Free Backbone for Vision with Kolmogorov-Arnold Networks

TL;DR

This work tackles the high computational cost and limited interpretability of attention in vision backbones by proposing Vision KAN (ViK), an attention-free architecture built upon Kolmogorov-Arnold networks. ViK replaces self-attention with a unified MultiPatch-RBFKAN token mixer that fuses patch-wise nonlinear RBF modeling, axis-wise separable mixing, and a low-rank global path to enable scalable long-range interactions. The approach achieves competitive ImageNet-1K accuracy with linear complexity, validated through extensive ablations and interpretability analyses of learned RBF mappings. Overall, ViK demonstrates that principled function-based token mixing can match or exceed attention-based performance while dramatically reducing computational overhead, offering a viable direction for efficient vision backbones.

Abstract

Attention mechanisms have become a key module in modern vision backbones due to their ability to model long-range dependencies. However, their quadratic complexity in sequence length and the difficulty of interpreting attention weights limit both scalability and clarity. Recent attention-free architectures demonstrate that strong performance can be achieved without pairwise attention, motivating the search for alternatives. In this work, we introduce Vision KAN (ViK), an attention-free backbone inspired by the Kolmogorov-Arnold Networks. At its core lies MultiPatch-RBFKAN, a unified token mixer that combines (a) patch-wise nonlinear transform with Radial Basis Function-based KANs, (b) axis-wise separable mixing for efficient local propagation, and (c) low-rank global mapping for long-range interaction. Employing as a drop-in replacement for attention modules, this formulation tackles the prohibitive cost of full KANs on high-resolution features by adopting a patch-wise grouping strategy with lightweight operators to restore cross-patch dependencies. Experiments on ImageNet-1K show that ViK achieves competitive accuracy with linear complexity, demonstrating the potential of KAN-based token mixing as an efficient and theoretically grounded alternative to attention.
Paper Structure (13 sections, 5 equations, 2 figures, 2 tables)

This paper contains 13 sections, 5 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Preliminaries: Kolmogorov-Arnold Networks
  4. Overall Architecture
  5. KAN-based Token Mixer
  6. Complexity Analysis
  7. Experiments and Results
  8. Experimental Setup
  9. Comparison with Representative Vision Backbones
  10. Ablation Study
  11. Visualization of Learned RBF Mappings
  12. Conclusion
  13. Acknowledgment

Figures (2)

  • Figure 1: Overview of the proposed Vision KAN (ViK). The backbone adopts a hierarchical design with four stages, where feature maps are progressively downsampled and processed by ViK blocks. Each ViK block contains the MultiPatch-RBFKAN module, which integrates (a) patch-wise nonlinear modeling with RBFKAN, (b) axis-wise separable depthwise convolutions for direction-sensitive local mixing, and (c) a low-rank global path for efficient long-range dependency modeling. Here, $\oplus$ and $\otimes$ denotes element-wise addition and multiplication, while ‖ denotes concatenation.
  • Figure 2: Examples of the univariate functions $\phi(x)$ learned by RBF across different stages of ViK. Shallow stages exhibit oscillatory nonlinearities, while deeper stages converge to smoother mappings, indicating progressive abstraction.