Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Are KANs Effective for Multivariate Time Series Forecasting?

Xiao Han, Xinfeng Zhang, Yiling Wu, Zhenduo Zhang, Zhe Wu

TL;DR

This paper investigates the effectiveness of Kolmogorov-Arnold Networks (KANs) for multivariate time-series forecasting and proposes the Multi-layer Mixture-of-KAN network (MMK). MMK employs a mixture-of-KAN (MoK) layer to adaptively route inputs to diverse KAN experts, preserving the ability to extract symbolic representations and thus interpretability. Across seven real-world datasets, MMK achieves strong or state-of-the-art results, demonstrates the ability to integrate into other models, and provides interpretability through gating weights and case studies. The work suggests that KANs offer a favorable balance between accuracy, efficiency, and interpretability in time-series forecasting and motivates further exploration of KAN-based architectures.

Abstract

Multivariate time series forecasting is a crucial task that predicts the future states based on historical inputs. Related techniques have been developing in parallel with the machine learning community, from early statistical learning methods to current deep learning methods. Despite their significant advancements, existing methods continue to struggle with the challenge of inadequate interpretability. The rise of the Kolmogorov-Arnold Network (KAN) provides a new perspective to solve this challenge, but current work has not yet concluded whether KAN is effective in time series forecasting tasks. In this paper, we aim to evaluate the effectiveness of KANs in time-series forecasting from the perspectives of performance, integrability, efficiency, and interpretability. To this end, we propose the Multi-layer Mixture-of-KAN network (MMK), which achieves excellent performance while retaining KAN's ability to be transformed into a combination of symbolic functions. The core module of MMK is the mixture-of-KAN layer, which uses a mixture-of-experts structure to assign variables to best-matched KAN experts. Then, we explore some useful experimental strategies to deal with the issues in the training stage. Finally, we compare MMK and various baselines on seven datasets. Extensive experimental and visualization results demonstrate that KANs are effective in multivariate time series forecasting. Code is available at: https://github.com/2448845600/EasyTSF.

Are KANs Effective for Multivariate Time Series Forecasting?

TL;DR

This paper investigates the effectiveness of Kolmogorov-Arnold Networks (KANs) for multivariate time-series forecasting and proposes the Multi-layer Mixture-of-KAN network (MMK). MMK employs a mixture-of-KAN (MoK) layer to adaptively route inputs to diverse KAN experts, preserving the ability to extract symbolic representations and thus interpretability. Across seven real-world datasets, MMK achieves strong or state-of-the-art results, demonstrates the ability to integrate into other models, and provides interpretability through gating weights and case studies. The work suggests that KANs offer a favorable balance between accuracy, efficiency, and interpretability in time-series forecasting and motivates further exploration of KAN-based architectures.

Abstract

Multivariate time series forecasting is a crucial task that predicts the future states based on historical inputs. Related techniques have been developing in parallel with the machine learning community, from early statistical learning methods to current deep learning methods. Despite their significant advancements, existing methods continue to struggle with the challenge of inadequate interpretability. The rise of the Kolmogorov-Arnold Network (KAN) provides a new perspective to solve this challenge, but current work has not yet concluded whether KAN is effective in time series forecasting tasks. In this paper, we aim to evaluate the effectiveness of KANs in time-series forecasting from the perspectives of performance, integrability, efficiency, and interpretability. To this end, we propose the Multi-layer Mixture-of-KAN network (MMK), which achieves excellent performance while retaining KAN's ability to be transformed into a combination of symbolic functions. The core module of MMK is the mixture-of-KAN layer, which uses a mixture-of-experts structure to assign variables to best-matched KAN experts. Then, we explore some useful experimental strategies to deal with the issues in the training stage. Finally, we compare MMK and various baselines on seven datasets. Extensive experimental and visualization results demonstrate that KANs are effective in multivariate time series forecasting. Code is available at: https://github.com/2448845600/EasyTSF.
Paper Structure (24 sections, 11 equations, 5 figures, 6 tables)

This paper contains 24 sections, 11 equations, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multivariate Time-Series Forecasting
  4. Kolmogorov-Arnold Network
  5. Multi-layer Mixture-of-KAN Network
  6. Problem Defintion
  7. Mixture-of-KAN Layer
  8. Network Architecture
  9. RevIN
  10. MoK Block
  11. Loss Function
  12. Training Strategy
  13. Experiments
  14. Experimental Settings
  15. Datasets
  16. ...and 9 more sections

Figures (5)

  • Figure 1: Two experiental findings on the ETTh1 dataset. Subfigure (a) shows the univariate forecasting results of KANs with different base functions, where results are minmax-normalized and 0 means the minimum MSE in the row. Subfigure (b) reports the training loss of one/three layers of Linear/KAN models.
  • Figure 2: The architecture of MMK. Subfigure (a) is the structure of the mixture-of-KAN (MoK) layer; subfigure (b) is the pre-sample initialization strategy for the last KAN layer.
  • Figure 3: The impact of different modules or training strategies.
  • Figure 4: Heatmap of the expert loads in the MoK on the Weather dataset. The value at location xy equal to 1 means that the gating network assigns the y-th variable to the x-th expert in all samples.
  • Figure 5: The weight of each input feature.