Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Review of Reinforcement Learning in Financial Applications

Yahui Bai, Yuhe Gao, Runzhe Wan, Sheng Zhang, Rui Song

TL;DR

This review presents a comprehensive study of the applications of RL in finance and conducts a series of meta-analyses to investigate the common themes in the literature, such as the factors that most significantly affect RL's performance compared with traditional methods.

Abstract

In recent years, there has been a growing trend of applying Reinforcement Learning (RL) in financial applications. This approach has shown great potential to solve decision-making tasks in finance. In this survey, we present a comprehensive study of the applications of RL in finance and conduct a series of meta-analyses to investigate the common themes in the literature, such as the factors that most significantly affect RL's performance compared to traditional methods. Moreover, we identify challenges including explainability, Markov Decision Process (MDP) modeling, and robustness that hinder the broader utilization of RL in the financial industry and discuss recent advancements in overcoming these challenges. Finally, we propose future research directions, such as benchmarking, contextual RL, multi-agent RL, and model-based RL to address these challenges and to further enhance the implementation of RL in finance.

A Review of Reinforcement Learning in Financial Applications

TL;DR

This review presents a comprehensive study of the applications of RL in finance and conducts a series of meta-analyses to investigate the common themes in the literature, such as the factors that most significantly affect RL's performance compared with traditional methods.

Abstract

In recent years, there has been a growing trend of applying Reinforcement Learning (RL) in financial applications. This approach has shown great potential to solve decision-making tasks in finance. In this survey, we present a comprehensive study of the applications of RL in finance and conduct a series of meta-analyses to investigate the common themes in the literature, such as the factors that most significantly affect RL's performance compared to traditional methods. Moreover, we identify challenges including explainability, Markov Decision Process (MDP) modeling, and robustness that hinder the broader utilization of RL in the financial industry and discuss recent advancements in overcoming these challenges. Finally, we propose future research directions, such as benchmarking, contextual RL, multi-agent RL, and model-based RL to address these challenges and to further enhance the implementation of RL in finance.

Paper Structure

This paper contains 33 sections, 2 equations, 5 figures, 3 tables.

Table of Contents

  1. Preliminary
  2. Applications of RL in Finance
  3. Summary of MDP Formulations in the Literature
  4. Applications in Market Making
  5. Single-agent RL in Market Making
  6. Multi-agent RL in Market Making
  7. Applications in Portfolio Management
  8. Bandit Algorithms for Portfolio Management
  9. RL Algorithms for Portfolio Management
  10. Complicated Policy Networks
  11. Empirical financial strategy oriented methods
  12. Methods aimed at enhancing robustness
  13. Market prediction based methods
  14. Applications in Optimal Execution
  15. Meta Analysis of Experimental Results
  16. ...and 18 more sections

Figures (5)

  • Figure 1: Frequency of the RL algorithms used in the literature.
  • Figure 2: RL Premium Analysis.
  • Figure 3: Analysis of realistic assumptions in surveyed papers.
  • Figure 4: Distribution of different state variables. Prices include the asset price, returns, and a combination; represents the agent's portfolio information; Pred. means the prediction results of stock movement and company news sentiment; Market Idx represents the market indicators like SP500; Tech. Idx means the technical indicators like Relative Strength Index (RSI); LOB means the limit-order-book is used as state; Time Index means the time interval is used as a state.
  • Figure 5: Impact of MDP Design on Model Performance.