Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Portfolio Management using Deep Reinforcement Learning

Ashish Anil Pawar, Vishnureddy Prashant Muskawar, Ritesh Tiku

TL;DR

This paper investigates portfolio management via deep reinforcement learning by reframing asset allocation as a weight-optimization task rather than discrete buy/hold/sell actions. It surveys prior DRL methods (CNN-based, attention-enhanced, double Q-learning, policy gradients) and then presents a weight-output DRL framework trained on 28 assets with states that incorporate prices, moving averages, and asset correlations. The proposed system uses a replay-buffered, Q-value–driven learning process to output a 28-dimensional weight vector that sums to one, evaluated in a postulated liquid market against conventional benchmarks. Results indicate superior risk-adjusted performance (notably the Sharpe ratio) across crypto and ETF portfolios, suggesting practical potential for DRL-driven portfolio optimization and setting the stage for incorporating more sophisticated RL architectures to handle market shocks.

Abstract

Algorithmic trading or Financial robots have been conquering the stock markets with their ability to fathom complex statistical trading strategies. But with the recent development of deep learning technologies, these strategies are becoming impotent. The DQN and A2C models have previously outperformed eminent humans in game-playing and robotics. In our work, we propose a reinforced portfolio manager offering assistance in the allocation of weights to assets. The environment proffers the manager the freedom to go long and even short on the assets. The weight allocation advisements are restricted to the choice of portfolio assets and tested empirically to knock benchmark indices. The manager performs financial transactions in a postulated liquid market without any transaction charges. This work provides the conclusion that the proposed portfolio manager with actions centered on weight allocations can surpass the risk-adjusted returns of conventional portfolio managers.

Portfolio Management using Deep Reinforcement Learning

TL;DR

This paper investigates portfolio management via deep reinforcement learning by reframing asset allocation as a weight-optimization task rather than discrete buy/hold/sell actions. It surveys prior DRL methods (CNN-based, attention-enhanced, double Q-learning, policy gradients) and then presents a weight-output DRL framework trained on 28 assets with states that incorporate prices, moving averages, and asset correlations. The proposed system uses a replay-buffered, Q-value–driven learning process to output a 28-dimensional weight vector that sums to one, evaluated in a postulated liquid market against conventional benchmarks. Results indicate superior risk-adjusted performance (notably the Sharpe ratio) across crypto and ETF portfolios, suggesting practical potential for DRL-driven portfolio optimization and setting the stage for incorporating more sophisticated RL architectures to handle market shocks.

Abstract

Algorithmic trading or Financial robots have been conquering the stock markets with their ability to fathom complex statistical trading strategies. But with the recent development of deep learning technologies, these strategies are becoming impotent. The DQN and A2C models have previously outperformed eminent humans in game-playing and robotics. In our work, we propose a reinforced portfolio manager offering assistance in the allocation of weights to assets. The environment proffers the manager the freedom to go long and even short on the assets. The weight allocation advisements are restricted to the choice of portfolio assets and tested empirically to knock benchmark indices. The manager performs financial transactions in a postulated liquid market without any transaction charges. This work provides the conclusion that the proposed portfolio manager with actions centered on weight allocations can surpass the risk-adjusted returns of conventional portfolio managers.
Paper Structure (21 sections, 9 figures)

This paper contains 21 sections, 9 figures.

Table of Contents

  1. INTRODUCTION
  2. Literature Survey
  3. Financial Portfolio Management
  4. Deep Convolution Neural Network Approach
  5. Deep Reinforcement Learning Approach
  6. Deep Attention Recurrent Q-Learning
  7. Double Q-Learning
  8. Policy Gradient Method
  9. High-Frequency Trading
  10. Proposed System
  11. Introduction
  12. The Data
  13. The Environment
  14. The Agent
  15. The Model
  16. ...and 6 more sections

Figures (9)

  • Figure 1: Portfolio Management System
  • Figure 2: A typical DDPG Flow Chart
  • Figure 3: Working of a DQN
  • Figure 4: RL Flow Chart
  • Figure 5: Performance of Min Variance Model
  • ...and 4 more figures