Deep Q-Network (DQN) multi-agent reinforcement learning (MARL) for Stock Trading

TL;DR

The study addresses automated stock trading under noisy, nonstationary market conditions by proposing a CNN-LSTM-DQN architecture that processes historical data formatted as a $9×9$ grid with $12$ feature channels to learn a Buy/Sell/Hold policy. The CNN extracts spatial patterns, the LSTM with Bahdanau attention captures temporal context, and the DQN learns a trading policy from a compact 17-dimensional state, with a target network and AdamW optimization to stabilize learning. Key contributions include the novel data representation, the integrated multi-network pipeline, and a comprehensive explainability suite that tracks saliency, attention, and gradient flows to validate learning progress. The work demonstrates the feasibility of a cohesive, multi-component RL trading agent and highlights its potential for generalization and risk-aware decision-making in real markets.

Abstract

This project addresses the challenge of automated stock trading, where traditional methods and direct reinforcement learning (RL) struggle with market noise, complexity, and generalization. Our proposed solution is an integrated deep learning framework combining a Convolutional Neural Network (CNN) to identify patterns in technical indicators formatted as images, a Long Short-Term Memory (LSTM) network to capture temporal dependencies across both price history and technical indicators, and a Deep Q-Network (DQN) agent which learns the optimal trading policy (buy, sell, hold) based on the features extracted by the CNN and LSTM.