Exploiting Risk-Aversion and Size-dependent fees in FX Trading with Fitted Natural Actor-Critic

TL;DR

This work focuses on the possibility of recognizing and leveraging intraday price patterns in the Foreign Exchange market, known for its extensive liquidity and flexibility, through the implementation of a Reinforcement Learning algorithm called Fitted Natural Actor-Critic.

Abstract

In recent years, the popularity of artificial intelligence has surged due to its widespread application in various fields. The financial sector has harnessed its advantages for multiple purposes, including the development of automated trading systems designed to interact autonomously with markets to pursue different aims. In this work, we focus on the possibility of recognizing and leveraging intraday price patterns in the Foreign Exchange market, known for its extensive liquidity and flexibility. Our approach involves the implementation of a Reinforcement Learning algorithm called Fitted Natural Actor-Critic. This algorithm allows the training of an agent capable of effectively trading by means of continuous actions, which enable the possibility of executing orders with variable trading sizes. This feature is instrumental to realistically model transaction costs, as they typically depend on the order size. Furthermore, it facilitates the integration of risk-averse approaches to induce the agent to adopt more conservative behavior. The proposed approaches have been empirically validated on EUR-USD historical data.

Figures (7)

• Figure 1: Expected cumulative returns obtained by the FNAC models trained in the discrete action space trading setting with different action persistence. Plots on the left and on the right show curves, respectively, for validation and test years. Performance is reported as mean percentages w.r.t. the invested amount (mean $\pm 2 stds$).
• Figure 2: Features importance of the FNAC models trained in the discrete action space trading setting with persistence equal to 5 (left) and 10 (right). The importance of a feature is defined as the normalized gain in terms of the Gini impurity index brought by that feature. The mean and the standard deviation of the importance of the ten most relevant features are reported.
• Figure 3: Expected cumulative returns obtained in test by the FNAC model trained in the continuous action space trading setting with fixed transaction fees. The performance of FNAC are compared with those obtained by FQI (trained in the discrete action space trading setting), PPO, and the baseline strategies Buy&Hold and Sell&Hold. Performance is reported as mean percentages w.r.t. the invested amount (mean $\pm$ 2 stds).
• Figure 4: Portfolio allocation selected during the test year by the FNAC models trained in the continuous action space trading setting with fixed transaction fees (left) and variable transaction fees (right). Each row of the heatmaps refers to a different trading day, whereas each column is specific for a trading minute.
• Figure 5: Expected cumulative returns obtained in test by the FNAC model trained in the continuous action space trading setting with variable transaction fees. The performance of FNAC are compared with those obtained by FQI (trained in the discrete action space trading setting), PPO, and the baseline strategies Buy&Hold and Sell&Hold. Performance is reported as mean percentages w.r.t. the invested amount (mean $\pm$ 2 stds).