Deep Reinforcement Learning for Solving the Fleet Size and Mix Vehicle Routing Problem

TL;DR

The paper tackles the Fleet Size and Mix Vehicle Routing Problem ($FSMVRP$), a challenging extension of VRP that jointly optimizes fleet composition and vehicle routing. It introduces a deep reinforcement learning approach, FRIPN, formulated as a Markov Decision Process and powered by a Transformer-based encoder–decoder with specialized embeddings, including a remaining graph embedding to guide fleet deployment. Through extensive experiments on synthetic and benchmark datasets, FRIPN achieves near-optimal solutions with substantial speed advantages, particularly in large-scale and time-constrained settings, and ablation confirms the value of the remaining graph embedding. The work demonstrates the practical potential of DRL for integrated fleet-and-route decisions and offers a scalable framework that can be extended to other FSMVRP variants and fleet configurations.

Abstract

The Fleet Size and Mix Vehicle Routing Problem (FSMVRP) is a prominent variant of the Vehicle Routing Problem (VRP), extensively studied in operations research and computational science. FSMVRP requires simultaneous decisions on fleet composition and routing, making it highly applicable to real-world scenarios such as short-term vehicle rental and on-demand logistics. However, these requirements also increase the complexity of FSMVRP, posing significant challenges, particularly in large-scale and time-constrained environments. In this paper, we propose a deep reinforcement learning (DRL)-based approach for solving FSMVRP, capable of generating near-optimal solutions within a few seconds. Specifically, we formulate the problem as a Markov Decision Process (MDP) and develop a novel policy network, termed FRIPN, that seamlessly integrates fleet composition and routing decisions. Our method incorporates specialized input embeddings designed for distinctdecision objectives, including a remaining graph embedding to facilitate effective vehicle employment decisions. Comprehensive experiments are conducted on both randomly generated instances and benchmark datasets. The experimental results demonstrate that our method exhibits notable advantages in terms of computational efficiency and scalability, particularly in large-scale and time-constrained scenarios. These strengths highlight the potential of our approach for practical applications and provide valuable inspiration for extending DRL-based techniques to other variants of VRP.

Figures (6)

• Figure 1: The example of the FSMVRP. Unlike the classic CVRP, FSMVRP requires consideration of both route planning and fleet composition. The inclusion of fleet composition introduces additional fixed costs, making it a critical factor that directly influences decision outcomes. Consequently, the need to optimize both routing and fleet structure renders FSMVRP more complex and challenging than the classic CVRP.
• Figure 2: The framework of our policy network. The encoder-decoder framework decides the current vehicle and node. Whether a vehicle departs from the depot determines whether its fixed cost is included in the reward calculation.
• Figure 3: The architecture of our policy network. The policy network mainly includes the node encoder, vehicle encoder, and decoder.
• Figure 4: The transform between employed and candidate vehicles. The type of vehicle is determined by its current location, and it also affects the encoding of its operational state.
• Figure 5: Ablation experiment results. Regardless of problem size, incorporating the remaining graph embedding consistently facilitates convergence to superior results.