Fleet Rebalancing for Expanding Shared e-Mobility Systems: A Multi-agent Deep Reinforcement Learning Approach
Man Luo, Bowen Du, Wenzhe Zhang, Tianyou Song, Kun Li, Hongming Zhu, Mark Birkin, Hongkai Wen
TL;DR
This paper tackles fleet rebalancing for expanding shared e-mobility systems under EV range and charging constraints by formulating the problem as a multi-agent reinforcement learning task. It introduces ac-PPO, a policy optimization method with action cascading that splits repositioning into inter-grid and intra-grid steps to handle non-stationary action spaces caused by continuous expansion, and couples it with a high-fidelity simulator trained on real-world data. The approach yields significant improvements in Demand Satisfied Rate and Net Revenue Value over baselines and prior MARL methods, and demonstrates robustness across different expansion speeds, charging times, and user incentive models. The work advances practical RL-based rebalancing for evolving urban mobility networks and provides a dataset-backed framework with potential for transfer to other cities, while acknowledging data-dependence and fairness considerations for future work.
Abstract
The electrification of shared mobility has become popular across the globe. Many cities have their new shared e-mobility systems deployed, with continuously expanding coverage from central areas to the city edges. A key challenge in the operation of these systems is fleet rebalancing, i.e., how EVs should be repositioned to better satisfy future demand. This is particularly challenging in the context of expanding systems, because i) the range of the EVs is limited while charging time is typically long, which constrain the viable rebalancing operations; and ii) the EV stations in the system are dynamically changing, i.e., the legitimate targets for rebalancing operations can vary over time. We tackle these challenges by first investigating rich sets of data collected from a real-world shared e-mobility system for one year, analyzing the operation model, usage patterns and expansion dynamics of this new mobility mode. With the learned knowledge we design a high-fidelity simulator, which is able to abstract key operation details of EV sharing at fine granularity. Then we model the rebalancing task for shared e-mobility systems under continuous expansion as a Multi-Agent Reinforcement Learning (MARL) problem, which directly takes the range and charging properties of the EVs into account. We further propose a novel policy optimization approach with action cascading, which is able to cope with the expansion dynamics and solve the formulated MARL. We evaluate the proposed approach extensively, and experimental results show that our approach outperforms the state-of-the-art, offering significant performance gain in both satisfied demand and net revenue.