HiPPO-KAN: Efficient KAN Model for Time Series Analysis

TL;DR

Surprisingly, although the HiPPO-KAN model keeps a constant parameter count as increasing window size, it significantly outperforms KAN model at larger window sizes, indicating that HiPPO-KAN offers significant parameter efficiency and scalability advantages for time series forecasting.

Abstract

In this study, we introduces a parameter-efficient model that outperforms traditional models in time series forecasting, by integrating High-order Polynomial Projection (HiPPO) theory into the Kolmogorov-Arnold network (KAN) framework. This HiPPO-KAN model achieves superior performance on long sequence data without increasing parameter count. Experimental results demonstrate that HiPPO-KAN maintains a constant parameter count while varying window sizes and prediction horizons, in contrast to KAN, whose parameter count increases linearly with window size. Surprisingly, although the HiPPO-KAN model keeps a constant parameter count as increasing window size, it significantly outperforms KAN model at larger window sizes. These results indicate that HiPPO-KAN offers significant parameter efficiency and scalability advantages for time series forecasting. Additionally, we address the lagging problem commonly encountered in time series forecasting models, where predictions fail to promptly capture sudden changes in the data. We achieve this by modifying the loss function to compute the MSE directly on the coefficient vectors in the HiPPO domain. This adjustment effectively resolves the lagging problem, resulting in predictions that closely follow the actual time series data. By incorporating HiPPO theory into KAN, this study showcases an efficient approach for handling long sequences with improved predictive accuracy, offering practical contributions for applications in large-scale time series data.

Figures (6)

• Figure 1: This diagram illustrates the process of encoding time series data using the HiPPO framework, transforming it with the Kolmogorov-Arnold Network (KAN), and decoding it back to the time domain. The initial time series $l$ is projected into a coefficient vector $c^{(L)}$ through HiPPO. This vector is then transformed by KAN into $c^{(L+1)}$, followed by decoding through HiPPO to reconstruct the time series of length $l+1$. This setup serves as an auto-encoder where HiPPO and KAN handle encoding, transformation, and decoding, respectively.
• Figure 2: HiPPO was applied to the S&P 500 data. The state space dimension used were $N=16,32,64,128,256$, and as $N$ increases, the approximation becomes increasingly closer to the original function, reflecting a higher fidelity representation of the underlying dynamics.
• Figure 3: MSE and MAE comparisons for various models (HiPPO-KAN, KAN, LSTM, RNN) using different window sizes (120, 500, 1200). The results show the performance of each model in terms of error metrics as the window size increases.
• Figure 4: Lagging Effect in KAN Models. These models exhibit a tendency to produce outputs that closely mimic the preceding values, indicating an inability to capture rapid changes in the data effectively.
• Figure 5: The modified loss function effectively resolves the lagging problem, resulting in predictions that closely follow the actual time series data. This result is based on a randomly selected segment of BTC-USDT 1-minute interval data, using a KAN architecture with a width of [16, 2, 16].