Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Embedded feature selection in LSTM networks with multi-objective evolutionary ensemble learning for time series forecasting

Raquel Espinosa, Fernando Jiménez, José Palma

TL;DR

This article presents a novel feature selection method embedded in Long Short-Term Memory networks, leveraging a multi-objective evolutionary algorithm that substantially improves the generalization ability of conventional LSTMs, effectively reducing overfitting.

Abstract

Time series forecasting plays a crucial role in diverse fields, necessitating the development of robust models that can effectively handle complex temporal patterns. In this article, we present a novel feature selection method embedded in Long Short-Term Memory networks, leveraging a multi-objective evolutionary algorithm. Our approach optimizes the weights and biases of the LSTM in a partitioned manner, with each objective function of the evolutionary algorithm targeting the root mean square error in a specific data partition. The set of non-dominated forecast models identified by the algorithm is then utilized to construct a meta-model through stacking-based ensemble learning. Furthermore, our proposed method provides an avenue for attribute importance determination, as the frequency of selection for each attribute in the set of non-dominated forecasting models reflects their significance. This attribute importance insight adds an interpretable dimension to the forecasting process. Experimental evaluations on air quality time series data from Italy and southeast Spain demonstrate that our method substantially improves the generalization ability of conventional LSTMs, effectively reducing overfitting. Comparative analyses against state-of-the-art CancelOut and EAR-FS methods highlight the superior performance of our approach.

Embedded feature selection in LSTM networks with multi-objective evolutionary ensemble learning for time series forecasting

TL;DR

This article presents a novel feature selection method embedded in Long Short-Term Memory networks, leveraging a multi-objective evolutionary algorithm that substantially improves the generalization ability of conventional LSTMs, effectively reducing overfitting.

Abstract

Time series forecasting plays a crucial role in diverse fields, necessitating the development of robust models that can effectively handle complex temporal patterns. In this article, we present a novel feature selection method embedded in Long Short-Term Memory networks, leveraging a multi-objective evolutionary algorithm. Our approach optimizes the weights and biases of the LSTM in a partitioned manner, with each objective function of the evolutionary algorithm targeting the root mean square error in a specific data partition. The set of non-dominated forecast models identified by the algorithm is then utilized to construct a meta-model through stacking-based ensemble learning. Furthermore, our proposed method provides an avenue for attribute importance determination, as the frequency of selection for each attribute in the set of non-dominated forecasting models reflects their significance. This attribute importance insight adds an interpretable dimension to the forecasting process. Experimental evaluations on air quality time series data from Italy and southeast Spain demonstrate that our method substantially improves the generalization ability of conventional LSTMs, effectively reducing overfitting. Comparative analyses against state-of-the-art CancelOut and EAR-FS methods highlight the superior performance of our approach.
Paper Structure (16 sections, 2 equations, 9 figures, 10 tables, 1 algorithm)

This paper contains 16 sections, 2 equations, 9 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Conclusions
  4. A multi-objective evolutionary ensemble learning approach for embedded feature selection in LSTM networks for time series forecasting
  5. General framework
  6. The multi-objective evolutionary algorithm
  7. Stacking-based ensemble learning
  8. Multi-step-ahead forecasting
  9. Feature importance
  10. Experiments and results
  11. Datasets
  12. Description of the experiments and results
  13. Analysis of results and discussion
  14. Conclusions and future work
  15. Abbreviations
  16. ...and 1 more sections

Figures (9)

  • Figure 1: General framework of EFS-LSTM-MOEA.
  • Figure 2: Embedded feature selection in an LSTM network.
  • Figure 3: Representation of an individual $\textbf{x}$.
  • Figure 4: Architecture of the conventional LSTM network for the Italy city air quality problem.
  • Figure 5: Architecture of the conventional LSTM network for the Lorca air quality problem.
  • ...and 4 more figures