Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Surge Routing: Event-informed Multiagent Reinforcement Learning for Autonomous Rideshare

Daniel Garces, Stephanie Gil

TL;DR

This work tackles surge-demand in urban ridesharing by fusing event-driven demand prediction from internet data with a scalable, event-informed, multiagent reinforcement learning routing framework. The approach comprises an event-processing and demand-prediction pipeline that uses sentence embeddings and spectral clustering to produce sector-level demand estimates, which are then mapped to intersections via occupancy-aware assignment for RL routing. A rollout-based, one-agent-at-a-time controller with limited sampling (certainty-equivalence) enables city-scale planning while keeping computation tractable. Experimental results on NYC HV-FHV data show substantial improvements in wait-time overhead (roughly 25-75% reductions) and increased serviced requests (about 1-4%), validating both the predictive and planning components and their integration in large-scale urban systems.

Abstract

Large events such as conferences, concerts and sports games, often cause surges in demand for ride services that are not captured in average demand patterns, posing unique challenges for routing algorithms. We propose a learning framework for an autonomous fleet of taxis that leverages event data from the internet to predict demand surges and generate cooperative routing policies. We achieve this through a combination of two major components: (i) a demand prediction framework that uses textual event information in the form of events' descriptions and reviews to predict event-driven demand surges over street intersections, and (ii) a scalable multiagent reinforcement learning framework that leverages demand predictions and uses one-agent-at-a-time rollout combined with limited sampling certainty equivalence to learn intersection-level routing policies. For our experimental results we consider real NYC ride share data for the year 2022 and information for more than 2000 events across 300 unique venues in Manhattan. We test our approach with a fleet of 100 taxis on a map with 2235 street intersections. Our experimental results demonstrate that our method learns routing policies that reduce wait time overhead per serviced request by 25% to 75%, while picking up 1% to 4% more requests than other model-based RL frameworks and classical methods in operations research.

Surge Routing: Event-informed Multiagent Reinforcement Learning for Autonomous Rideshare

TL;DR

This work tackles surge-demand in urban ridesharing by fusing event-driven demand prediction from internet data with a scalable, event-informed, multiagent reinforcement learning routing framework. The approach comprises an event-processing and demand-prediction pipeline that uses sentence embeddings and spectral clustering to produce sector-level demand estimates, which are then mapped to intersections via occupancy-aware assignment for RL routing. A rollout-based, one-agent-at-a-time controller with limited sampling (certainty-equivalence) enables city-scale planning while keeping computation tractable. Experimental results on NYC HV-FHV data show substantial improvements in wait-time overhead (roughly 25-75% reductions) and increased serviced requests (about 1-4%), validating both the predictive and planning components and their integration in large-scale urban systems.

Abstract

Large events such as conferences, concerts and sports games, often cause surges in demand for ride services that are not captured in average demand patterns, posing unique challenges for routing algorithms. We propose a learning framework for an autonomous fleet of taxis that leverages event data from the internet to predict demand surges and generate cooperative routing policies. We achieve this through a combination of two major components: (i) a demand prediction framework that uses textual event information in the form of events' descriptions and reviews to predict event-driven demand surges over street intersections, and (ii) a scalable multiagent reinforcement learning framework that leverages demand predictions and uses one-agent-at-a-time rollout combined with limited sampling certainty equivalence to learn intersection-level routing policies. For our experimental results we consider real NYC ride share data for the year 2022 and information for more than 2000 events across 300 unique venues in Manhattan. We test our approach with a fleet of 100 taxis on a map with 2235 street intersections. Our experimental results demonstrate that our method learns routing policies that reduce wait time overhead per serviced request by 25% to 75%, while picking up 1% to 4% more requests than other model-based RL frameworks and classical methods in operations research.
Paper Structure (31 sections, 2 equations, 7 figures, 4 tables)

This paper contains 31 sections, 2 equations, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Dynamic Vehicle Routing
  4. Demand prediction
  5. Problem Formulation
  6. Environment
  7. Requests
  8. State representation and control space
  9. Rollout-based routing
  10. Problem
  11. Our Approach
  12. Rollout-based scalable routing framework
  13. Event-informed demand estimation
  14. Estimating the probability distribution for the number of requests $\tilde{p}_{\eta_{s_k}}$
  15. Event data and sentence embeddings
  16. ...and 16 more sections

Figures (7)

  • Figure 1: Motivating example
  • Figure 2: General System Overview showing our proposed approach's four modules: 1) the event processing module, 2) the demand prediction module, 3) the demand assignment module, and 4) the model-based RL routing module.
  • Figure 3: Our routing environment over 2235 intersections in NYC
  • Figure 4: Example of one-agent-at-a-time rollout with two agents, where each agent only has two available actions.
  • Figure 5: Mapping probability distributions over sectors to intersections for pickup and drop-off. The potential number of riders at each intersection is estimated using occupancy schedules and the maximum occupancy of the locale
  • ...and 2 more figures