FilterNet: Harnessing Frequency Filters for Time Series Forecasting

TL;DR

Inspired by the filtering process, a novel perspective of enlightening signal processing for deep time series forecasting is explored, namely FilterNet, built upon the proposed learnable frequency filters to extract key informative temporal patterns by selectively passing or attenuating certain components of time series signals.

Abstract

While numerous forecasters have been proposed using different network architectures, the Transformer-based models have state-of-the-art performance in time series forecasting. However, forecasters based on Transformers are still suffering from vulnerability to high-frequency signals, efficiency in computation, and bottleneck in full-spectrum utilization, which essentially are the cornerstones for accurately predicting time series with thousands of points. In this paper, we explore a novel perspective of enlightening signal processing for deep time series forecasting. Inspired by the filtering process, we introduce one simple yet effective network, namely FilterNet, built upon our proposed learnable frequency filters to extract key informative temporal patterns by selectively passing or attenuating certain components of time series signals. Concretely, we propose two kinds of learnable filters in the FilterNet: (i) Plain shaping filter, that adopts a universal frequency kernel for signal filtering and temporal modeling; (ii) Contextual shaping filter, that utilizes filtered frequencies examined in terms of its compatibility with input signals for dependency learning. Equipped with the two filters, FilterNet can approximately surrogate the linear and attention mappings widely adopted in time series literature, while enjoying superb abilities in handling high-frequency noises and utilizing the whole frequency spectrum that is beneficial for forecasting. Finally, we conduct extensive experiments on eight time series forecasting benchmarks, and experimental results have demonstrated our superior performance in terms of both effectiveness and efficiency compared with state-of-the-art methods. Code is available at this repository: https://github.com/aikunyi/FilterNet

Figures (13)

• Figure 1: Performance of Mean Squared Error (MSE) on a simple synthetic multi-frequency signal. More details about the experimental settings can be found in Appendix \ref{['experimental_signal_details']}.
• Figure 2: The overall architecture of FilterNet. (i) Instance normalization is employed to address the non-stationarity among time series data; (ii) The frequency filter block is applied to capture the temporal patterns, which has two different implementations, i.e., plain shaping filter and contextual shaping filter; (iii) Feed-forward network is adopted to project the temporal patterns extracted by frequency filter block back onto the time series data and make predictions.
• Figure 3: The structure of frequency filters. (a) Plain shaping filter: the plain shaping filter is initialized randomly with channel-shared (left) or channel-unique (right) parameters, and then performs circular convolution (i.e., the symbol $\circledast$) with the input time series; (b) Contextual shaping filter: the contextual shaping filter firstly learns a data-dependent filter and then conducts multiplication (i.e., the symbol $\odot$) with the frequency representation of the input time series.
• Figure 4: Predictions produced by FilterNet on trend and multi-periodic signals with noises. When adding noises for interference, FilterNet can perform more robust forecasting than iTransformer itransformer_2024.
• Figure 5: Visualization of prediction on the ETTh1 dataset with lookback and horizon length as 96.