Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Vehicle Routing with Finite Time Horizon using Deep Reinforcement Learning with Improved Network Embedding

Ayan Maity, Sudeshna Sarkar

TL;DR

This work tackles the Vehicle Routing Problem with a Finite Time Horizon (VRP-FTH) by introducing a routing network embedding that captures both local graph structure and a horizon-aware global context. It combines Graph Attention Networks with Edge Features (GAT-Edge) and a Cross-Attention global embedding to form a rich state representation for a policy-gradient RL agent, enabling more informed routing decisions under finite time constraints. Empirical results on real-world EMA/Vienna networks and synthetic Euclidean networks show substantial improvements in customer service rates and significantly faster solution times compared with traditional optimization and existing RL methods, with ablations demonstrating the value of edge features, global context, and horizon integration. The proposed framework is applicable to VRP-FTH and can be extended to other VRP variants, offering a scalable, context-aware approach that better reflects real-world routing networks.

Abstract

In this paper, we study the vehicle routing problem with a finite time horizon. In this routing problem, the objective is to maximize the number of customer requests served within a finite time horizon. We present a novel routing network embedding module which creates local node embedding vectors and a context-aware global graph representation. The proposed Markov decision process for the vehicle routing problem incorporates the node features, the network adjacency matrix and the edge features as components of the state space. We incorporate the remaining finite time horizon into the network embedding module to provide a proper routing context to the embedding module. We integrate our embedding module with a policy gradient-based deep Reinforcement Learning framework to solve the vehicle routing problem with finite time horizon. We trained and validated our proposed routing method on real-world routing networks, as well as synthetically generated Euclidean networks. Our experimental results show that our method achieves a higher customer service rate than the existing routing methods. Additionally, the solution time of our method is significantly lower than that of the existing methods.

Vehicle Routing with Finite Time Horizon using Deep Reinforcement Learning with Improved Network Embedding

TL;DR

This work tackles the Vehicle Routing Problem with a Finite Time Horizon (VRP-FTH) by introducing a routing network embedding that captures both local graph structure and a horizon-aware global context. It combines Graph Attention Networks with Edge Features (GAT-Edge) and a Cross-Attention global embedding to form a rich state representation for a policy-gradient RL agent, enabling more informed routing decisions under finite time constraints. Empirical results on real-world EMA/Vienna networks and synthetic Euclidean networks show substantial improvements in customer service rates and significantly faster solution times compared with traditional optimization and existing RL methods, with ablations demonstrating the value of edge features, global context, and horizon integration. The proposed framework is applicable to VRP-FTH and can be extended to other VRP variants, offering a scalable, context-aware approach that better reflects real-world routing networks.

Abstract

In this paper, we study the vehicle routing problem with a finite time horizon. In this routing problem, the objective is to maximize the number of customer requests served within a finite time horizon. We present a novel routing network embedding module which creates local node embedding vectors and a context-aware global graph representation. The proposed Markov decision process for the vehicle routing problem incorporates the node features, the network adjacency matrix and the edge features as components of the state space. We incorporate the remaining finite time horizon into the network embedding module to provide a proper routing context to the embedding module. We integrate our embedding module with a policy gradient-based deep Reinforcement Learning framework to solve the vehicle routing problem with finite time horizon. We trained and validated our proposed routing method on real-world routing networks, as well as synthetically generated Euclidean networks. Our experimental results show that our method achieves a higher customer service rate than the existing routing methods. Additionally, the solution time of our method is significantly lower than that of the existing methods.
Paper Structure (25 sections, 13 equations, 3 figures, 5 tables)

This paper contains 25 sections, 13 equations, 3 figures, 5 tables.

Table of Contents

  1. Abstract
  2. Introduction
  3. Problem Definition
  4. Related Work
  5. Traditional Optimization-based Routing Methods
  6. Traditional Methods for VRP with Finite Horizon.
  7. Traditional Methods for Other VRP Variants.
  8. Reinforcement Learning-based Routing Methods
  9. RL Methods for VRP with Finite Horizon.
  10. RL Methods for Other VRP Variants.
  11. Methodology
  12. Markov Decision Process
  13. Routing Network Embedding Module
  14. Graph Attention Networks with Edge Features (GAT-Edge).
  15. Global Graph Embedding using Cross-Attention Mechanism.
  16. ...and 10 more sections

Figures (3)

  • Figure 1: Vehicle Routing with Finite Time Horizon
  • Figure 2: Policy Network with GAT-based routing network embedding module (PG-GAT-Edge)
  • Figure 3: Customer Service Rate (%) under different time horizon ($U$, in hours) for PG-GAT-Edge (Our method) and DRL-Transformer tang2023energy on Vienna-160 and EN-50