CNN-DRL with Shuffled Features in Finance

TL;DR

The paper tackles improving deep reinforcement learning for stock trading by reformatting the input feature vector into a 2D matrix suitable for CNN processing through a shuffled-feature permutation. It uses a FinRL-based environment and PPO to compare three agents (MLP, original CNN, and shuffled CNN), showing the shuffled CNN yields higher cumulative rewards, especially during 2020–2023. The work demonstrates that arranging related features adjacently enhances the CNN's ability to extract temporal and cross-asset patterns, offering a practical preprocessing step that boosts performance without changing the learning algorithm. The findings suggest strong potential for extending CNN-DRL approaches to higher-frequency data and broader asset universes in quantitative finance.

Abstract

In prior methods, it was observed that the application of Convolutional Neural Networks agent in Deep Reinforcement Learning to financial data resulted in an enhanced reward. In this study, a specific permutation was applied to the feature vector, thereby generating a CNN matrix that strategically positions more pertinent features in close proximity. Our comprehensive experimental evaluations unequivocally demonstrate a substantial enhancement in reward attainment.

Figures (5)

• Figure 1: DRL agent and environment.
• Figure 2: Sliding window to create CNN input matrix channel, before shuffling.
• Figure 3: Sliding window to create CNN input matrix channel, after shuffling.
• Figure 4: Our CNN architecture.
• Figure 5: Training reward.