Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Kernel Corrector LSTM

Rodrigo Tuna, Yassine Baghoussi, Carlos Soares, João Mendes-Moreira

TL;DR

The paper tackles data quality issues in time-series forecasting by adopting Read-Write ML, where the model can modify data during learning. It introduces Kernel Corrector LSTM (KcLSTM), which replaces the SARIMA-based meta-learner in cLSTM with Gaussian Kernel Smoothing to reduce computational cost while preserving predictive accuracy. Empirical results on the M4 Monthly subset show that KcLSTM often outperforms cLSTM and can surpass LSTM in statistically significant wins, though tuning and data sensitivity affect performance. The work demonstrates a faster, data-correcting forecasting approach with practical implications for real-time or data-imperfect scenarios, and suggests exploring other estimators in future work.

Abstract

Forecasting methods are affected by data quality issues in two ways: 1. they are hard to predict, and 2. they may affect the model negatively when it is updated with new data. The latter issue is usually addressed by pre-processing the data to remove those issues. An alternative approach has recently been proposed, Corrector LSTM (cLSTM), which is a Read \& Write Machine Learning (RW-ML) algorithm that changes the data while learning to improve its predictions. Despite promising results being reported, cLSTM is computationally expensive, as it uses a meta-learner to monitor the hidden states of the LSTM. We propose a new RW-ML algorithm, Kernel Corrector LSTM (KcLSTM), that replaces the meta-learner of cLSTM with a simpler method: Kernel Smoothing. We empirically evaluate the forecasting accuracy and the training time of the new algorithm and compare it with cLSTM and LSTM. Results indicate that it is able to decrease the training time while maintaining a competitive forecasting accuracy.

Kernel Corrector LSTM

TL;DR

The paper tackles data quality issues in time-series forecasting by adopting Read-Write ML, where the model can modify data during learning. It introduces Kernel Corrector LSTM (KcLSTM), which replaces the SARIMA-based meta-learner in cLSTM with Gaussian Kernel Smoothing to reduce computational cost while preserving predictive accuracy. Empirical results on the M4 Monthly subset show that KcLSTM often outperforms cLSTM and can surpass LSTM in statistically significant wins, though tuning and data sensitivity affect performance. The work demonstrates a faster, data-correcting forecasting approach with practical implications for real-time or data-imperfect scenarios, and suggests exploring other estimators in future work.

Abstract

Forecasting methods are affected by data quality issues in two ways: 1. they are hard to predict, and 2. they may affect the model negatively when it is updated with new data. The latter issue is usually addressed by pre-processing the data to remove those issues. An alternative approach has recently been proposed, Corrector LSTM (cLSTM), which is a Read \& Write Machine Learning (RW-ML) algorithm that changes the data while learning to improve its predictions. Despite promising results being reported, cLSTM is computationally expensive, as it uses a meta-learner to monitor the hidden states of the LSTM. We propose a new RW-ML algorithm, Kernel Corrector LSTM (KcLSTM), that replaces the meta-learner of cLSTM with a simpler method: Kernel Smoothing. We empirically evaluate the forecasting accuracy and the training time of the new algorithm and compare it with cLSTM and LSTM. Results indicate that it is able to decrease the training time while maintaining a competitive forecasting accuracy.
Paper Structure (15 sections, 6 equations, 4 figures, 2 tables)

This paper contains 15 sections, 6 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LSTM
  4. Data-Centric Time Series Forecasting
  5. Corrector LSTM
  6. Kernel Corrector LSTM
  7. Training
  8. Data Correction
  9. Experimental Setup
  10. Datasets
  11. Evaluation Metrics
  12. Results
  13. Comparison with cLSTM
  14. Comparison with LSTM
  15. Conclusion

Figures (4)

  • Figure 1: Cell unit of the LSTM recurent neural network lstmpic.
  • Figure 2: Gaussian Smoothing of the Hidden States, represented as a series.
  • Figure 3: Example where data reconstruction was successful.
  • Figure 4: Example where the data reconstruction destroyed the data which caused the model not to capture the information of the series.