FSMLP: Modelling Channel Dependencies With Simplex Theory Based Multi-Layer Perceptions In Frequency Domain

TL;DR

FSMLP tackles overfitting in channel-mix time-series forecasting by constraining inter-channel MLP weights to lie on a standard $n$-simplex via Simplex-MLP, reducing model capacity and memorization of noise. Built on SCWM and FTM blocks, FSMLP operates in the frequency domain to model inter-channel dependencies across periods, while optimizing both time-domain MSE and frequency-domain MAE losses. Theoretical analysis shows a lower Rademacher complexity bound for Simplex-MLP, and experiments on seven benchmarks demonstrate superior accuracy and efficiency, with strong scalability and transferability to other channel-wise MLP methods. The approach offers a practical, robust solution for long-horizon TSF in diverse domains such as energy, web data, and weather forecasting.

Abstract

Time series forecasting (TSF) plays a crucial role in various domains, including web data analysis, energy consumption prediction, and weather forecasting. While Multi-Layer Perceptrons (MLPs) are lightweight and effective for capturing temporal dependencies, they are prone to overfitting when used to model inter-channel dependencies. In this paper, we investigate the overfitting problem in channel-wise MLPs using Rademacher complexity theory, revealing that extreme values in time series data exacerbate this issue. To mitigate this issue, we introduce a novel Simplex-MLP layer, where the weights are constrained within a standard simplex. This strategy encourages the model to learn simpler patterns and thereby reducing overfitting to extreme values. Based on the Simplex-MLP layer, we propose a novel \textbf{F}requency \textbf{S}implex \textbf{MLP} (FSMLP) framework for time series forecasting, comprising of two kinds of modules: \textbf{S}implex \textbf{C}hannel-\textbf{W}ise MLP (SCWM) and \textbf{F}requency \textbf{T}emporal \textbf{M}LP (FTM). The SCWM effectively leverages the Simplex-MLP to capture inter-channel dependencies, while the FTM is a simple yet efficient temporal MLP designed to extract temporal information from the data. Our theoretical analysis shows that the upper bound of the Rademacher Complexity for Simplex-MLP is lower than that for standard MLPs. Moreover, we validate our proposed method on seven benchmark datasets, demonstrating significant improvements in forecasting accuracy and efficiency, while also showcasing superior scalability. Additionally, we demonstrate that Simplex-MLP can improve other methods that use channel-wise MLP to achieve less overfitting and improved performance. Code are available \href{https://github.com/FMLYD/FSMLP}{\textcolor{red}{here}}.

Figures (3)

• Figure 1: Comparison of overfitting behavior among FSMLP, TimesNet, TSMixer, and Autoformer on the ETTh1 dataset, with both look-back window and prediction length set to 96. TSMixer, TimesNet, and Autoformer show a rapid decline in training loss, but their validation loss remains high, indicating overfitting. In contrast, FSMLP maintains lower validation loss, demonstrating that the Simplex MLP effectively mitigates the overfitting issues typically encountered by MLPs when modeling channel dependencies.
• Figure 2: The overall architecture of the proposed FSMLP. We first extract the inter-channel dependencies with our Simplex Channel-Wise MLP then use the Frequency Temporal MLP to extract temporal dependencies.
• Figure 3: This is the efficiency comparison of the proposed FSMLP with baselines on ETTh1 dataset with $L=96$ and $\tau=96$.

Theorems & Definitions (2)

• Theorem 1
• proof