Kernel-U-Net: Multivariate Time Series Forecasting using Custom Kernels

TL;DR

Kernel-U-Net introduces a kernel-customizable U-Net for multivariate time series forecasting by partitioning inputs into patches and applying layer-wise configurable kernels, enabling expressive modeling with controlled, near-linear computational costs when higher layers use Transformer/LSTM modules. The architecture is a symmetric encoder–decoder with skip connections, where kernels operate on patches and a Kernel Wrapper standardizes kernel usage. Extensive experiments on seven public datasets show Kernel-U-Net either matches or exceeds state-of-the-art results (e.g., NLinear, PatchTST) in most multivariate settings, while delivering notable computation and memory savings. The work proposes a flexible framework for rapid kernel experimentation and practical large-scale forecasting applications, with public release of the codebase.

Abstract

Time series forecasting task predicts future trends based on historical information. Transformer-based U-Net architectures, despite their success in medical image segmentation, have limitations in both expressiveness and computation efficiency in time series forecasting as evidenced in YFormer. To tackle these challenges, we introduce Kernel-U-Net, a flexible and kernel-customizable U-shape neural network architecture. The kernel-U-Net encoder compresses the input series into latent vectors, and its symmetric decoder subsequently expands these vectors into output series. Specifically, Kernel-U-Net separates the procedure of partitioning input time series into patches from kernel manipulation, thereby providing the convenience of customized executing kernels. Our method offers two primary advantages: 1) Flexibility in kernel customization to adapt to specific datasets; and 2) Enhanced computational efficiency, with the complexity of the Transformer layer reduced to linear. Experiments on seven real-world datasets, demonstrate that Kernel-U-Net's performance either exceeds or meets that of the existing state-of-the-art model in the majority of cases in channel-independent settings. The source code for Kernel-U-Net will be made publicly available for further research and application.

Figures (6)

• Figure 1: Illustration of Kernel U-Net Architecture. a) Architecture of Kernel U-Net, it allows executing linear kernel and nonlinear kernels such as MLP, LSTM, and Transformer. b) Illustration of K-U-Net Encoder, the application of $\phi^{(l)}_{enc}$ on patches is independent of the choice of kernel. c) In the K-U-Net decoder, a custom kernel $\phi^{(l)}_{dec}$ expands the vectors into patches in the reverse order.
• Figure 2: Structure of Linear Kernel. a) Linear Kernel, b) Multi-layer perceptron (MLP) Kernel with Tanh activation, c) LSTM Kernel, d) Transformer Kernel
• Figure 3: Search result of K-U-Net with Linear, MLP, LSTM, Transformer kernels on ETTh1 dataset.
• Figure 4: GPU consumation of K-U-Net, PatchTST and Yformer.
• Figure 5: Search result of K-U-Net with linear, MLP, LSTM, Transformer kernels on Weather dataset.