Collaboration! Towards Robust Neural Methods for Routing Problems

TL;DR

An ensemble-based Collaborative Neural Framework w.r.t. the defense of neural VRP methods, which is crucial yet underexplored in the literature, and achieves impressive out-of-distribution generalization on benchmark instances.

Abstract

Despite enjoying desirable efficiency and reduced reliance on domain expertise, existing neural methods for vehicle routing problems (VRPs) suffer from severe robustness issues -- their performance significantly deteriorates on clean instances with crafted perturbations. To enhance robustness, we propose an ensemble-based Collaborative Neural Framework (CNF) w.r.t. the defense of neural VRP methods, which is crucial yet underexplored in the literature. Given a neural VRP method, we adversarially train multiple models in a collaborative manner to synergistically promote robustness against attacks, while boosting standard generalization on clean instances. A neural router is designed to adeptly distribute training instances among models, enhancing overall load balancing and collaborative efficacy. Extensive experiments verify the effectiveness and versatility of CNF in defending against various attacks across different neural VRP methods. Notably, our approach also achieves impressive out-of-distribution generalization on benchmark instances.

Figures (8)

• Figure 1: (a-b) Performance of POMO kwon2020pomo on TSP100 against the attacker in zhang2022learning. The value in brackets denotes the number of trained models. We report the average optimality (opt.) gap over 1000 test instances. (c) Solution visualizations on an adversarial instance. These results reveal the vulnerability of existing neural methods to adversarial attacks, and the existence of undesirable trade-off between standard generalization (a) and adversarial robustness (b) in VRPs. Details of the attacker and experimental setups can be found in Appendix \ref{['app:attack_perturb']} and Section \ref{['exps']}, respectively.
• Figure 2: The overview of CNF. Suppose we train $M=3$ models ($\Theta=\{\theta_1,\theta_2,\theta_3\}$) on a batch ($B=3$) of clean instances. The inner maximization generates local ($\tilde{x}$) and global ($\bar{x}$) adversarial instances within $T$ steps. In the outer minimization, a neural router $\theta_r$ is jointly trained to distribute instances to the $M$ models for training. Specifically, based on the logit matrix $\mathcal{P}$ predicted by the neural router, each model selects the instances with Top$\mathcal{K}$-largest logits (e.g., red ones). The neural router is optimized to maximize the improvement of collaborative performance after each training step of $\Theta$. For simplicity, we omit the superscripts of instances in the outer minimization.
• Figure 3: Ablation studies on TSP100. The metrics of Uniform and Fixed Adv. are reported.
• Figure 4: An illustration of generated adversarial instances (i.e., the grey ones). (a) An adversarial instance generated by zhang2022learning on CVRP, where the triangle represents the depot node. A deeper color denotes a heavier node demand; (b) An adversarial instance generated by geisler2022generalization on TSP, where the red nodes represent the newly inserted adversarial nodes; (c) An adversarial instance generated by lu2023roco on asymmetric TSP, where the cost of an edge is in half.
• Figure 5: Left panel: Performance of each model $\theta_j \in \Theta$ in CNF ($M=3$), and the overall collaboration performance of $\Theta$. Right panel: A demonstration (i.e., attention map) of the learned routing policy for $\theta_0$. The horizontal axis is the index of the training instance. Concretely, 0-2: clean instances $x$; 3-11: local adversarial instances $\tilde{x}$; 12-14: global adversarial instances $\bar{x}$. The vertical axis is the epoch of the checkpoint. A deeper color represents a higher probability to be selected.