Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing the Cross-Size Generalization for Solving Vehicle Routing Problems via Continual Learning

Jingwen Li, Zhiguang Cao, Yaoxin Wu, Tang Liu

TL;DR

The paper tackles cross-size generalization in VRPs by proposing a model-agnostic continual learning framework that trains on ascending problem sizes using experience replay and two exemplar-based regularization schemes (inter-task and intra-task) to preserve past knowledge while adapting to larger instances. The approach improves cross-size generalization for autoregressive VRP solvers and outperforms state-of-the-art baselines designed for generalizability, with ablations confirming the synergy of components. Its model-agnostic nature preserves inference efficiency, making it practical for real-world VRP applications. The work offers a solid pathway to scalable, generalizable learning-based VRP solvers and points to large-scale VRP extensions and cross-distribution generalization as future directions.

Abstract

Exploring machine learning techniques for addressing vehicle routing problems has attracted considerable research attention. To achieve decent and efficient solutions, existing deep models for vehicle routing problems are typically trained and evaluated using instances of a single size. This substantially limits their ability to generalize across different problem sizes and thus hampers their practical applicability. To address the issue, we propose a continual learning based framework that sequentially trains a deep model with instances of ascending problem sizes. Specifically, on the one hand, we design an inter-task regularization scheme to retain the knowledge acquired from smaller problem sizes in the model training on a larger size. On the other hand, we introduce an intra-task regularization scheme to consolidate the model by imitating the latest desirable behaviors during training on each size. Additionally, we exploit the experience replay to revisit instances of formerly trained sizes for mitigating the catastrophic forgetting. Experimental results show that our approach achieves predominantly superior performance across various problem sizes (either seen or unseen in the training), as compared to state-of-the-art deep models including the ones specialized for generalizability enhancement. Meanwhile, the ablation studies on the key designs manifest their synergistic effect in the proposed framework.

Enhancing the Cross-Size Generalization for Solving Vehicle Routing Problems via Continual Learning

TL;DR

The paper tackles cross-size generalization in VRPs by proposing a model-agnostic continual learning framework that trains on ascending problem sizes using experience replay and two exemplar-based regularization schemes (inter-task and intra-task) to preserve past knowledge while adapting to larger instances. The approach improves cross-size generalization for autoregressive VRP solvers and outperforms state-of-the-art baselines designed for generalizability, with ablations confirming the synergy of components. Its model-agnostic nature preserves inference efficiency, making it practical for real-world VRP applications. The work offers a solid pathway to scalable, generalizable learning-based VRP solvers and points to large-scale VRP extensions and cross-distribution generalization as future directions.

Abstract

Exploring machine learning techniques for addressing vehicle routing problems has attracted considerable research attention. To achieve decent and efficient solutions, existing deep models for vehicle routing problems are typically trained and evaluated using instances of a single size. This substantially limits their ability to generalize across different problem sizes and thus hampers their practical applicability. To address the issue, we propose a continual learning based framework that sequentially trains a deep model with instances of ascending problem sizes. Specifically, on the one hand, we design an inter-task regularization scheme to retain the knowledge acquired from smaller problem sizes in the model training on a larger size. On the other hand, we introduce an intra-task regularization scheme to consolidate the model by imitating the latest desirable behaviors during training on each size. Additionally, we exploit the experience replay to revisit instances of formerly trained sizes for mitigating the catastrophic forgetting. Experimental results show that our approach achieves predominantly superior performance across various problem sizes (either seen or unseen in the training), as compared to state-of-the-art deep models including the ones specialized for generalizability enhancement. Meanwhile, the ablation studies on the key designs manifest their synergistic effect in the proposed framework.

Paper Structure

This paper contains 19 sections, 5 equations, 2 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Deep models for VRPs.
  4. Cross-size generalization.
  5. Continual learning.
  6. Preliminaries and Notations
  7. VRP Statement
  8. Autoregressive deep models for VRPs
  9. Methodology
  10. Experience replay
  11. Regularization schemes
  12. Training algorithm
  13. Experiments
  14. Training setups.
  15. Inference setups.
  16. ...and 4 more sections

Figures (2)

  • Figure 1: The illustration of the proposed framework with inter-task regularization. For each mini-batch training during current task interval, we employ 1) experience replay to sample a size from formerly trained sizes and current one, and generate instances with that sampled size; 2) inter-task regularization to foster the current model to emulate an exemplary model for knowledge retention.
  • Figure 2: Regularization with two exemplar model updating strategies. (a) inter-task: exemplar model is updated after training on a whole task; (b) intra-task: exemplar model is updated multiple times during training on a task for concentrating more on newly encountered (larger) size.