Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Task Vehicle Routing Solver via Mixture of Specialized Experts under State-Decomposable MDP

Yuxin Pan, Zhiguang Cao, Chengyang Gu, Liu Liu, Peilin Zhao, Yize Chen, Fangzhen Lin

TL;DR

This paper tackles multi-variant VRPs by exploiting a compositional structure through basis VRP variants. It introduces State-Decomposable MDP (SDMDP) to decompose states into a Cartesian product of basis states and derives LS-SDMDP for latent policy reuse via a deterministic mixture function, establishing theoretical equivalences between unified and basis policies. The Mixture-of-Specialized-Experts Solver (MoSES) implements basis policies as LoRA adapters and uses an adaptive gating mechanism to dynamically fuse experts, enabling efficient handling of 16 VRP variants with diverse constraints. Empirically, MoSES outperforms state-of-the-art unified solvers and demonstrates robust generalization and favorable trade-offs between performance and compute. The work highlights a scalable path to reuse specialized Basis solvers within a unified framework, with practical impact for multi-task VRP applications.

Abstract

Existing neural methods for multi-task vehicle routing problems (VRPs) typically learn unified solvers to handle multiple constraints simultaneously. However, they often underutilize the compositional structure of VRP variants, each derivable from a common set of basis VRP variants. This critical oversight causes unified solvers to miss out the potential benefits of basis solvers, each specialized for a basis VRP variant. To overcome this limitation, we propose a framework that enables unified solvers to perceive the shared-component nature across VRP variants by proactively reusing basis solvers, while mitigating the exponential growth of trained neural solvers. Specifically, we introduce a State-Decomposable MDP (SDMDP) that reformulates VRPs by expressing the state space as the Cartesian product of basis state spaces associated with basis VRP variants. More crucially, this formulation inherently yields the optimal basis policy for each basis VRP variant. Furthermore, a Latent Space-based SDMDP extension is developed by incorporating both the optimal basis policies and a learnable mixture function to enable the policy reuse in the latent space. Under mild assumptions, this extension provably recovers the optimal unified policy of SDMDP through the mixture function that computes the state embedding as a mapping from the basis state embeddings generated by optimal basis policies. For practical implementation, we introduce the Mixture-of-Specialized-Experts Solver (MoSES), which realizes basis policies through specialized Low-Rank Adaptation (LoRA) experts, and implements the mixture function via an adaptive gating mechanism. Extensive experiments conducted across VRP variants showcase the superiority of MoSES over prior methods.

Multi-Task Vehicle Routing Solver via Mixture of Specialized Experts under State-Decomposable MDP

TL;DR

This paper tackles multi-variant VRPs by exploiting a compositional structure through basis VRP variants. It introduces State-Decomposable MDP (SDMDP) to decompose states into a Cartesian product of basis states and derives LS-SDMDP for latent policy reuse via a deterministic mixture function, establishing theoretical equivalences between unified and basis policies. The Mixture-of-Specialized-Experts Solver (MoSES) implements basis policies as LoRA adapters and uses an adaptive gating mechanism to dynamically fuse experts, enabling efficient handling of 16 VRP variants with diverse constraints. Empirically, MoSES outperforms state-of-the-art unified solvers and demonstrates robust generalization and favorable trade-offs between performance and compute. The work highlights a scalable path to reuse specialized Basis solvers within a unified framework, with practical impact for multi-task VRP applications.

Abstract

Existing neural methods for multi-task vehicle routing problems (VRPs) typically learn unified solvers to handle multiple constraints simultaneously. However, they often underutilize the compositional structure of VRP variants, each derivable from a common set of basis VRP variants. This critical oversight causes unified solvers to miss out the potential benefits of basis solvers, each specialized for a basis VRP variant. To overcome this limitation, we propose a framework that enables unified solvers to perceive the shared-component nature across VRP variants by proactively reusing basis solvers, while mitigating the exponential growth of trained neural solvers. Specifically, we introduce a State-Decomposable MDP (SDMDP) that reformulates VRPs by expressing the state space as the Cartesian product of basis state spaces associated with basis VRP variants. More crucially, this formulation inherently yields the optimal basis policy for each basis VRP variant. Furthermore, a Latent Space-based SDMDP extension is developed by incorporating both the optimal basis policies and a learnable mixture function to enable the policy reuse in the latent space. Under mild assumptions, this extension provably recovers the optimal unified policy of SDMDP through the mixture function that computes the state embedding as a mapping from the basis state embeddings generated by optimal basis policies. For practical implementation, we introduce the Mixture-of-Specialized-Experts Solver (MoSES), which realizes basis policies through specialized Low-Rank Adaptation (LoRA) experts, and implements the mixture function via an adaptive gating mechanism. Extensive experiments conducted across VRP variants showcase the superiority of MoSES over prior methods.
Paper Structure (25 sections, 4 theorems, 15 equations, 13 figures, 9 tables)

This paper contains 25 sections, 4 theorems, 15 equations, 13 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Methodology
  5. State-Decomposable MDP
  6. Latent Space-based SDMDP
  7. Mixture-of-Specialized-Experts Solver
  8. Experiments
  9. Empirical Results
  10. Hyperparameter Studies
  11. Ablation Studies
  12. Conclusions and Limitations
  13. Environment Details
  14. VRP Variants
  15. Problem Instance Generation
  16. ...and 10 more sections

Key Result

Theorem 1

The optimal unified policy $\pi^{\ast}$ and the $i$-th optimal basis policy $\pi^{(i)\ast}$ coincide in their value functions for each state $s_{t}$ associated with the $i$-th basis task: $V^{\pi^{\ast}, \mathcal{P}}(s_{t}) = V^{\pi^{(i)\ast},\mathcal{P}}(s_{t})$. Furthermore, if both the optimal un

Figures (13)

  • Figure 1: Left: SDMDP employs the policy that incorporates a mixture function. Right: LS-SDMDP integrates both the optimal basis policies and the mixture function.
  • Figure 2: \ref{['fig:curve-1']}\ref{['fig:curve-2']} compare Gated-LoRA against standard LoRA with varying $\beta\in(0,1]$. \ref{['fig:curve-3']}\ref{['fig:curve-4']} investigate the effect of LoRA ranks on the model performance.
  • Figure 3: Activations.
  • Figure 4: Routing Methods.
  • Figure 5: This figure investigates the effect of LoRA ranks on MoSES(RF) and MoSES(CaDA) under both $N=50$ and $N=100$ settings.
  • ...and 8 more figures

Theorems & Definitions (8)

  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Theorem 4
  • proof