Diverse Approaches to Optimal Execution Schedule Generation

TL;DR

The paper presents a calibrated, RL-driven framework for optimal execution that blends empirically grounded transient market impact with PPO-based policies and quality-diversity via MAP-Elites. By operating in a Gymnasium-based GEO environment calibrated to real minute-bar data, the authors demonstrate that a CNN-powered PPO agent achieves substantial reductions in arrival slippage and total cost relative to VWAP and TWAP baselines, while MAP-Elites reveals regime-specific specialists with meaningful gains in certain liquidity-volatility regimes. However, significant generalisation challenges across market regimes and high computational demands for robust specialist development temper the enthusiasm for immediate deployment. The work highlights the potential of regime-adaptive execution and points to ensemble/routing approaches that leverage specialist strengths while fall-backing to robust baselines in uncertain regimes.

Abstract

We present the first application of MAP-Elites, a quality-diversity algorithm, to trade execution. Rather than searching for a single optimal policy, MAP-Elites generates a diverse portfolio of regime-specialist strategies indexed by liquidity and volatility conditions. Individual specialists achieve 8-10% performance improvements within their behavioural niches, while other cells show degradation, suggesting opportunities for ensemble approaches that combine improved specialists with the baseline PPO policy. Results indicate that quality-diversity methods offer promise for regime-adaptive execution, though substantial computational resources per behavioural cell may be required for robust specialist development across all market conditions. To ensure experimental integrity, we develop a calibrated Gymnasium environment focused on order scheduling rather than tactical placement decisions. The simulator features a transient impact model with exponential decay and square-root volume scaling, fit to 400+ U.S. equities with R^2>0.02 out-of-sample. Within this environment, two Proximal Policy Optimization architectures - both MLP and CNN feature extractors - demonstrate substantial improvements over industry baselines, with the CNN variant achieving 2.13 bps arrival slippage versus 5.23 bps for VWAP on 4,900 out-of-sample orders ($21B notional). These results validate both the simulation realism and provide strong single-policy baselines for quality-diversity methods.

Figures (15)

• Figure 1: Evolution of optimal execution approaches: from classical models to empirical impact models to reinforcement learning, with this work positioned at the intersection of empirically calibrated models and RL methods.
• Figure 2: Illustrative transient impact kernels in basis points. The instantaneous kernel is shown schematically as a unit impulse; exponential and power-law forms capture transient, decaying impact with different memory.
• Figure 3: Classic RL Flow diagram, adapted from sutton2018reinforcement.
• Figure 4: Schematic overview of the GEO environment architecture, showing the interaction between agent, environment, and calibrated market impact model.
• Figure 5: Sample set of generated orders with impact and decay coefficients, stratified by symbol, size, and time horizon.