Amplifier: Bringing Attention to Neglected Low-Energy Components in Time Series Forecasting

TL;DR

This paper tackles the persistent problem that time series forecasting models undervalue low-energy components in the spectral domain. It introduces an energy amplification technique implemented via an energy amplification block and an energy restoration block, and couples it with a semi-channel interaction (SCI) block and a seasonal-trend forecaster to form the Amplifier model. The approach yields consistent improvements across eight real-world benchmarks, achieving state-of-the-art accuracy with competitive efficiency, and demonstrates that the energy amplification technique can be generalized to existing backbones beyond Amplifier. The work provides both theoretical motivation and extensive empirical evidence that paying attention to low-energy components enhances forecasting performance in diverse domains.

Abstract

We propose an energy amplification technique to address the issue that existing models easily overlook low-energy components in time series forecasting. This technique comprises an energy amplification block and an energy restoration block. The energy amplification block enhances the energy of low-energy components to improve the model's learning efficiency for these components, while the energy restoration block returns the energy to its original level. Moreover, considering that the energy-amplified data typically displays two distinct energy peaks in the frequency spectrum, we integrate the energy amplification technique with a seasonal-trend forecaster to model the temporal relationships of these two peaks independently, serving as the backbone for our proposed model, Amplifier. Additionally, we propose a semi-channel interaction temporal relationship enhancement block for Amplifier, which enhances the model's ability to capture temporal relationships from the perspective of the commonality and specificity of each channel in the data. Extensive experiments on eight time series forecasting benchmarks consistently demonstrate our model's superiority in both effectiveness and efficiency compared to state-of-the-art methods.

Figures (7)

• Figure 1: Analysis about low-energy components in time series forecasting. (a) Indispensability: Discarding low-energy components results in an increased MSE value. (b) Dependence on energy magnitude: The components that are ignored are consistently those with low energy, irrespective of their position within the frequency bands.
• Figure 2: The overall architecture of Amplifier. (i) The energy amplification block aims to increase the energy of low-energy components, while energy restoration block performs the inverse operation of the energy amplification block; (ii) SCI block is employed to capture both the common temporal pattern and specific temporal pattern; (iii) Seasonal-Trend Forecaster is then utilized to decompose seasonal and trend information, and then make predictions.
• Figure 3: Ablation prediction results of the energy amplification technique in four representative models.
• Figure 4: Model efficiency comparison in terms of MSE, the scale of parameters, and training speed.
• Figure 5: Prediction spectrums of both Amplifier and four SOTA models on two synthetic signals. The small green circles represent the neglected low-energy components.

Theorems & Definitions (6)

• Theorem 1
• Theorem 2
• Definition 1
• Lemma 1
• Theorem 1
• Theorem 2