Improving DeFi Accessibility through Efficient Liquidity Provisioning with Deep Reinforcement Learning

TL;DR

The paper addresses profitability and accessibility of liquidity provisioning in Uniswap v3 given concentrated liquidity and impermanent loss. It models LP decision-making as an MDP and solves it with Proximal Policy Optimization, including a reward that balances fee income and LVR-impermanent loss costs. It uses a rolling-window data-driven training/testing regime and demonstrates that an active LP driven by DRL outperforms a heuristic passive strategy in a majority of out-of-sample periods. The approach aims to make DeFi markets more accessible to smaller retail participants through more efficient, on-chain liquidity management.

Abstract

This paper applies deep reinforcement learning (DRL) to optimize liquidity provisioning in Uniswap v3, a decentralized finance (DeFi) protocol implementing an automated market maker (AMM) model with concentrated liquidity. We model the liquidity provision task as a Markov Decision Process (MDP) and train an active liquidity provider (LP) agent using the Proximal Policy Optimization (PPO) algorithm. The agent dynamically adjusts liquidity positions by using information about price dynamics to balance fee maximization and impermanent loss mitigation. We use a rolling window approach for training and testing, reflecting realistic market conditions and regime shifts. This study compares the data-driven performance of the DRL-based strategy against common heuristics adopted by small retail LP actors that do not systematically modify their liquidity positions. By promoting more efficient liquidity management, this work aims to make DeFi markets more accessible and inclusive for a broader range of participants. Through a data-driven approach to liquidity management, this work seeks to contribute to the ongoing development of more efficient and user-friendly DeFi markets.

Figures (5)

• Figure 1: Price series of WETH/USDC and corresponding sliding windows: The figure illustrates the hourly price series of the WETH/USDC token pair in the Uniswap pool with a 0.05% fee tier, highlighting the sliding windows used for training and testing the RL model.
• Figure 2: Comparison of active and passive LP strategies when $x_0=2$: The figure compares the cumulative rewards achieved by the active and passive LP strategies over the out-of-sample testing period. The bar plot in the bottom panel visually represents the performance differences, corresponding to the numerical values displayed in the table at the top.
• Figure 3: The figure presents four panels showing the WETH/USDC price dynamics in the Uniswap pool with a 0.05% fee. The first column illustrates the price paths under a passive LP strategy, which rebalances at three predefined windows during the out-of-sample test period. The second column displays the equivalent price dynamics for the active LP strategy. The two windows correspond to the periods referenced in Fig. \ref{['Fig:cumrew_table']}, ending on 2022-05-14 (left column) and 2023-07-26 (right column).
• Figure 4: The two panels compare the cumulative rewards achieved by the active and passive LP strategies during the out-of-sample periods corresponding to the testing windows ending on 2022-05-14 (left panel) and 2023-07-26 (right panel). The plots illustrate the performance differences over time, emphasizing the active LP's ability to adapt to market dynamics and achieve higher rewards in varying conditions.
• Figure 5: Comparison of active and passive LP strategies when $x_0=10$: The figure compares the cumulative rewards achieved by the active and passive LP strategies over the out-of-sample testing period. The bar plot in the bottom panel visually represents the performance differences, corresponding to the numerical values displayed in the table at the top.