Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimization

Srijan Sood, Kassiani Papasotiriou, Marius Vaiciulis, Tucker Balch

TL;DR

This work is a thorough comparison between model-free DRL and MVO for optimal portfolio allocation, and details the specifics of how to make DRL for portfolio optimization work in practice, also noting the adjustments needed for MVO.

Abstract

Portfolio Management is the process of overseeing a group of investments, referred to as a portfolio, with the objective of achieving predetermined investment goals. Portfolio optimization is a key component that involves allocating the portfolio assets so as to maximize returns while minimizing risk taken. It is typically carried out by financial professionals who use a combination of quantitative techniques and investment expertise to make decisions about the portfolio allocation. Recent applications of Deep Reinforcement Learning (DRL) have shown promising results when used to optimize portfolio allocation by training model-free agents on historical market data. Many of these methods compare their results against basic benchmarks or other state-of-the-art DRL agents but often fail to compare their performance against traditional methods used by financial professionals in practical settings. One of the most commonly used methods for this task is Mean-Variance Portfolio Optimization (MVO), which uses historical time series information to estimate expected asset returns and covariances, which are then used to optimize for an investment objective. Our work is a thorough comparison between model-free DRL and MVO for optimal portfolio allocation. We detail the specifics of how to make DRL for portfolio optimization work in practice, also noting the adjustments needed for MVO. Backtest results demonstrate strong performance of the DRL agent across many metrics, including Sharpe ratio, maximum drawdowns, and absolute returns.

Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimization

TL;DR

This work is a thorough comparison between model-free DRL and MVO for optimal portfolio allocation, and details the specifics of how to make DRL for portfolio optimization work in practice, also noting the adjustments needed for MVO.

Abstract

Portfolio Management is the process of overseeing a group of investments, referred to as a portfolio, with the objective of achieving predetermined investment goals. Portfolio optimization is a key component that involves allocating the portfolio assets so as to maximize returns while minimizing risk taken. It is typically carried out by financial professionals who use a combination of quantitative techniques and investment expertise to make decisions about the portfolio allocation. Recent applications of Deep Reinforcement Learning (DRL) have shown promising results when used to optimize portfolio allocation by training model-free agents on historical market data. Many of these methods compare their results against basic benchmarks or other state-of-the-art DRL agents but often fail to compare their performance against traditional methods used by financial professionals in practical settings. One of the most commonly used methods for this task is Mean-Variance Portfolio Optimization (MVO), which uses historical time series information to estimate expected asset returns and covariances, which are then used to optimize for an investment objective. Our work is a thorough comparison between model-free DRL and MVO for optimal portfolio allocation. We detail the specifics of how to make DRL for portfolio optimization work in practice, also noting the adjustments needed for MVO. Backtest results demonstrate strong performance of the DRL agent across many metrics, including Sharpe ratio, maximum drawdowns, and absolute returns.
Paper Structure (23 sections, 14 equations, 4 figures, 2 tables)

This paper contains 23 sections, 14 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Mean-Variance Portfolio Optimization
  5. Reinforcement Learning
  6. RL for Portfolio Allocation
  7. Problem Setup
  8. Actions
  9. States
  10. Reward
  11. Learning Algorithm
  12. Policy Optimization
  13. RL Environment Specifics
  14. Experiments
  15. Data & Features
  16. ...and 8 more sections

Figures (4)

  • Figure 1: S&P 500 and its 11 sector indices between 2006 and 2021.
  • Figure 2: Backtest Results: MVO vs DRL Portfolio Allocation.
  • Figure 3: a) DRL Monthly Returns b) DRL Annual Returns c) DRL Monthly Distribution of Returns.
  • Figure 4: a) MVO Monthly Returns b) MVO Annual Returns c) MVO Monthly Distribution of Returns.