SpectralKAN: Weighted Activation Distribution Kolmogorov-Arnold Network for Hyperspectral Image Change Detection

TL;DR

High-dimensional hyperspectral change detection suffers from redundant activations and information loss when applying traditional Kolmogorov-Arnold networks (KANs). The authors address this by introducing Weighted Activation Distribution KANs (WKANs) and a Multilevel Tensor Splitting Framework (MTSF), enabling efficient, dimension-wise feature extraction. They instantiate SpectralKAN for CD, integrating two WKANs in a tensor-splitting pipeline to achieve strong accuracy with dramatically reduced parameters, FLOPs, memory, and latency across five datasets. This approach provides a practical, scalable solution for high-dimensional change detection and suggests future extensions to frequency-domain processing and other high-dimensional tasks.

Abstract

Kolmogorov-Arnold networks (KANs) represent data features by learning the activation functions and demonstrate superior accuracy with fewer parameters, FLOPs, GPU memory usage (Memory), shorter training time (TraT), and testing time (TesT) when handling low-dimensional data. However, when applied to high-dimensional data, which contains significant redundant information, the current activation mechanism of KANs leads to unnecessary computations, thereby reducing computational efficiency. KANs require reshaping high-dimensional data into a one-dimensional tensor as input, which inevitably results in the loss of dimensional information. To address these limitations, we propose weighted activation distribution KANs (WKANs), which reduce the frequency of activations per node and distribute node information into different output nodes through weights to avoid extracting redundant information. Furthermore, we introduce a multilevel tensor splitting framework (MTSF), which decomposes high-dimensional data to extract features from each dimension independently and leverages tensor-parallel computation to significantly improve the computational efficiency of WKANs on high-dimensional data. In this paper, we design SpectralKAN for hyperspectral image change detection using the proposed MTSF. SpectralKAN demonstrates outstanding performance across five datasets, achieving an overall accuracy (OA) of 0.9801 and a Kappa coefficient (K) of 0.9514 on the Farmland dataset, with only 8 k parameters, 0.07 M FLOPs, 911 MB Memory, 13.26 S TraT, and 2.52 S TesT, underscoring its superior accuracy-efficiency trade-off. The source code is publicly available at https://github.com/yanhengwang-heu/SpectralKAN.

Figures (11)

• Figure 1: Performance comparison of state-of-the-art methods and the proposed SpectralKAN on the commonly used Farmland dataset.
• Figure 2: Flowchart of the SpectralKAN. SpectralKAN first splits each patch into multiple spatial local tensors $\{v_i\}_{i=1}^{h\times w}$. Global spatial features f are then extracted using spatial-level WKANs within the MTSF. We further split f into channel local tensors $\{f_e\}_{e=1}^{b}$. Channel-level WKANs in MTSF subsequently extract spectral features and classify them as either changed or unchanged.
• Figure 3: Structure of the l-th WKAN layer with m input nodes and n output nodes.
• Figure 4: Structure of a single KAN layer with one input node and n output nodes.
• Figure 5: The results on Farmland datasets. (a) Before temporal hyperspectral images, (b) After temporal hyperspectral images, (c) Groundtruth, (d) ML-EDAN, (e) SST-Former, (f) CSANet, (g) DA-Former, (h) TriTF, (i) HyperSIGMA, (j) Ours. The white pixels are changed, and the black pixels are unchanged.