Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TraveLLM: Could you plan my new public transit route in face of a network disruption?

Bowen Fang, Zixiao Yang, Xuan Di

TL;DR

TraveLLM tackles disruption-aware public transit routing by leveraging LLMs to interpret natural language queries and multimodal map data to generate practical travel plans under dynamic disruptions. The authors propose a two-stage architecture (planner and summary) with prompt engineering, and evaluate GPT-4, Claude 3, and Gemini across a benchmark of disruption scenarios, measuring with $M_{conn}$, $M_{avoid}$, $M_{time}$, and $M_{transfers}$. Results show GPT-4 achieves strong connectivity and avoidance alongside competitive timing, with map imagery and a separate summary agent further improving plan quality and output reliability ($M_{format}$). The work demonstrates a viable path toward more flexible, user-specific navigation systems that integrate real-time disruption information and multimodal data, laying a foundation for future enhancements in visual reasoning and real-world deployment.

Abstract

Existing navigation systems often fail during urban disruptions, struggling to incorporate real-time events and complex user constraints, such as avoiding specific areas. We address this gap with TraveLLM, a system using Large Language Models (LLMs) for disruption-aware public transit routing. We leverage LLMs' reasoning capabilities to directly process multimodal user queries combining natural language requests (origin, destination, preferences, disruption info) with map data (e.g., subway, bus, bike-share). To evaluate this approach, we design challenging test scenarios reflecting real-world disruptions like weather events, emergencies, and dynamic service availability. We benchmark the performance of state-of-the-art LLMs, including GPT-4, Claude 3, and Gemini, on generating accurate travel plans. Our experiments demonstrate that LLMs, notably GPT-4, can effectively generate viable and context-aware navigation plans under these demanding conditions. These findings suggest a promising direction for using LLMs to build more flexible and intelligent navigation systems capable of handling dynamic disruptions and diverse user needs.

TraveLLM: Could you plan my new public transit route in face of a network disruption?

TL;DR

TraveLLM tackles disruption-aware public transit routing by leveraging LLMs to interpret natural language queries and multimodal map data to generate practical travel plans under dynamic disruptions. The authors propose a two-stage architecture (planner and summary) with prompt engineering, and evaluate GPT-4, Claude 3, and Gemini across a benchmark of disruption scenarios, measuring with , , , and . Results show GPT-4 achieves strong connectivity and avoidance alongside competitive timing, with map imagery and a separate summary agent further improving plan quality and output reliability (). The work demonstrates a viable path toward more flexible, user-specific navigation systems that integrate real-time disruption information and multimodal data, laying a foundation for future enhancements in visual reasoning and real-world deployment.

Abstract

Existing navigation systems often fail during urban disruptions, struggling to incorporate real-time events and complex user constraints, such as avoiding specific areas. We address this gap with TraveLLM, a system using Large Language Models (LLMs) for disruption-aware public transit routing. We leverage LLMs' reasoning capabilities to directly process multimodal user queries combining natural language requests (origin, destination, preferences, disruption info) with map data (e.g., subway, bus, bike-share). To evaluate this approach, we design challenging test scenarios reflecting real-world disruptions like weather events, emergencies, and dynamic service availability. We benchmark the performance of state-of-the-art LLMs, including GPT-4, Claude 3, and Gemini, on generating accurate travel plans. Our experiments demonstrate that LLMs, notably GPT-4, can effectively generate viable and context-aware navigation plans under these demanding conditions. These findings suggest a promising direction for using LLMs to build more flexible and intelligent navigation systems capable of handling dynamic disruptions and diverse user needs.
Paper Structure (17 sections, 4 figures, 4 tables)

This paper contains 17 sections, 4 figures, 4 tables.

Table of Contents

  1. INTRODUCTION
  2. Limitations of Current Planning Methods
  3. The Emerging Role of Large Language Models
  4. METHODOLOGY
  5. TraveLLM Framework Overview
  6. Prompt Engineering Strategy
  7. EXPERIMENTS
  8. Implementation Details
  9. Benchmark Scenarios
  10. Evaluation Metrics
  11. RESULTS
  12. LLM Planner Comparison
  13. Ablation Studies
  14. Importance of Map Images
  15. Efficacy of Separate Summary Agent
  16. ...and 2 more sections

Figures (4)

  • Figure 1: System architecture for the TraveLLM prototype, showing the two-stage process from user query to structured plan.
  • Figure 2: Visual avoidance constraint for Scenario S5: avoid the black rectangle.
  • Figure 3: Visual input for multimodal Scenario S6: user location (circle) and Citi Bike availability (bubbles).
  • Figure 4: Example of diverse route generation by TraveLLM. For a trip from Columbia University to 110th St station, distinct routes optimizing for safety (left, red), efficiency (middle, blue), and scenery (right, green) are generated based on qualitative criteria interpreted by the LLM Planner ($\mathcal{L}_{planner}$). Route justifications are derived from planner output.