Self-Supervised Learning for Time Series: A Review & Critique of FITS

TL;DR

This work achieves the best results of any known open-source model on multivariate regression and promising results in multiple/linear regression on price datasets, on top of vastly improving upon what FITS achieves as a standalone model.

Abstract

Accurate time series forecasting is a highly valuable endeavour with applications across many industries. Despite recent deep learning advancements, increased model complexity, and larger model sizes, many state-of-the-art models often perform worse or on par with simpler models. One of those cases is a recently proposed model, FITS, claiming competitive performance with significantly reduced parameter counts. By training a one-layer neural network in the complex frequency domain, we are able to replicate these results. Our experiments on a wide range of real-world datasets further reveal that FITS especially excels at capturing periodic and seasonal patterns, but struggles with trending, non-periodic, or random-resembling behavior. With our two novel hybrid approaches, where we attempt to remedy the weaknesses of FITS by combining it with DLinear, we achieve the best results of any known open-source model on multivariate regression and promising results in multiple/linear regression on price datasets, on top of vastly improving upon what FITS achieves as a standalone model.

Figures (37)

• Figure 1: FITS pipeline for reconstruction task
• Figure 2: The effect of different cutoff frequencies using a low pass and high pass filter
• Figure 3: FITS trained with a cutoff at the 5th harmonic of a signal of mixed sines at higher harmonics with a dominant period of 60. The left most plots show the original and reconstructed/extrapolated signal with the vertical line visualising the split between input and prediction. Middle plots are the frequency representations of the signal and the reconstruction. Right plot is the absolute value of the complex weight matrix of FITS for this data showing high activations corresponding to the harmonics.
• Figure 4: Inference of FITS on the harmonic base components illustrating the inherent separation of the frequency components of the signal.
• Figure 5: Increasing the length of the period of the signal that is being trained on to longer than the output allows. Output and original signal to the left and weights for each of the models to the right illustrating spectral leakage.