Resolving Latency and Inventory Risk in Market Making with Reinforcement Learning
Junzhe Jiang, Chang Yang, Xinrun Wang, Zhiming Li, Xiao Huang, Bo Li
TL;DR
This work tackles latency and inventory risk in market making by introducing Relaver, a latency-aware RL framework. It builds a realistic MM environment with random order-delivery delays and batch matching, and augments the RL state-action space to include order hold times. Relaver combines a dynamic-programming–driven Q-teacher to guide RL exploration and a pretrained trend prediction expert to adjust inventory exposure under evolving market conditions. Across four real-world Chinese index option datasets, Relaver outperforms baselines on profitability and risk metrics, while ablation studies highlight the complementary value of the Q-teacher and trend predictor. The approach also provides a standardized, open-source simulation platform to advance reproducible, latency-aware MM research at realistic operational scales.
Abstract
The latency of the exchanges in Market Making (MM) is inevitable due to hardware limitations, system processing times, delays in receiving data from exchanges, the time required for order transmission to reach the market, etc. Existing reinforcement learning (RL) methods for Market Making (MM) overlook the impact of these latency, which can lead to unintended order cancellations due to price discrepancies between decision and execution times and result in undesired inventory accumulation, exposing MM traders to increased market risk. Therefore, these methods cannot be applied in real MM scenarios. To address these issues, we first build a realistic MM environment with random delays of 30-100 milliseconds for order placement and market information reception, and implement a batch matching mechanism that collects orders within every 500 milliseconds before matching them all at once, simulating the batch auction mechanisms adopted by some exchanges. Then, we propose Relaver, an RL-based method for MM to tackle the latency and inventory risk issues. The three main contributions of Relaver are: i) we introduce an augmented state-action space that incorporates order hold time alongside price and volume, enabling Relaver to optimize execution strategies under latency constraints and time-priority matching mechanisms, ii) we leverage dynamic programming (DP) to guide the exploration of RL training for better policies, iii) we train a market trend predictor, which can guide the agent to intelligently adjust the inventory to reduce the risk. Extensive experiments and ablation studies on four real-world datasets demonstrate that \textsc{Relaver} significantly improves the performance of state-of-the-art RL-based MM strategies across multiple metrics.