Gradient Boosting Decision Tree with LSTM for Investment Prediction

TL;DR

This work tackles the challenge of financial time-series forecasting by proposing a stacking ensemble that fuses CatBoost, LightGBM, and LSTM, with a two-layer LSTM meta-learner. By leveraging the strengths of gradient-boosting trees for non-linear patterns and LSTM for sequential dependencies, the approach delivers substantial performance gains, reporting an $R^2$ around $0.815$ and lower MAE/MSE/RMSE compared to individual models and classic baselines. The study demonstrates that hybrid ensemble methods can reduce errors during market changes and offers a flexible framework for integrating additional techniques. Practically, this indicates a viable path toward more accurate and robust investment decision-support systems driven by ensemble learning.

Abstract

This paper proposes a hybrid framework combining LSTM (Long Short-Term Memory) networks with LightGBM and CatBoost for stock price prediction. The framework processes time-series financial data and evaluates performance using seven models: Artificial Neural Networks (ANNs), Convolutional Neural Networks (CNNs), Bidirectional LSTM (BiLSTM), vanilla LSTM, XGBoost, LightGBM, and standard Neural Networks (NNs). Key metrics, including MAE, R-squared, MSE, and RMSE, are used to establish benchmarks across different time scales. Building on these benchmarks, we develop an ensemble model that combines the strengths of sequential and tree-based approaches. Experimental results show that the proposed framework improves accuracy by 10 to 15 percent compared to individual models and reduces error during market changes. This study highlights the potential of ensemble methods for financial forecasting and provides a flexible design for integrating new machine learning techniques.