Learning-Based TSP-Solvers Tend to Be Overly Greedy

TL;DR

The paper investigates why learning-based TSP solvers often rely on greedy heuristics by introducing the nearest-neighbor density ρ_n to quantify the extent to which optimal tours align with nearest-neighbor relations. It shows that across random uniform, random normal, and real-world TSPLib instances, ρ_n remains high, suggesting a persistent greedy bias, and it provides an asymptotic lower bound ρ_n ≥ (27−32β)/7 with β ≈ 0.7124. To combat this bias, the authors propose distribution-shift and perturbation-based data augmentation, including scale-free network and drilling-pattern instances, and demonstrate that fine-tuning with augmented data improves generalization on diverse tests. However, they prove fundamental limits: there is no efficient complete generator based on ρ_n unless NP = coNP, and no efficient algorithmic coverage by a polynomially sized solver ensemble unless NP = P, implying that universal neural solvers for TSP are unlikely. The work emphasizes the need for realistic benchmarks and interpretable features to advance AI-powered combinatorial optimization, rather than pursuing universal solvers via data augmentation alone.

Abstract

Deep learning has shown significant potential in solving combinatorial optimization problems such as the Euclidean traveling salesman problem (TSP). However, most training and test instances for existing TSP algorithms are generated randomly from specific distributions like uniform distribution. This has led to a lack of analysis and understanding of the performance of deep learning algorithms in out-of-distribution (OOD) generalization scenarios, which has a close relationship with the worst-case performance in the combinatorial optimization field. For data-driven algorithms, the statistical properties of randomly generated datasets are critical. This study constructs a statistical measure called nearest-neighbor density to verify the asymptotic properties of randomly generated datasets and reveal the greedy behavior of learning-based solvers, i.e., always choosing the nearest neighbor nodes to construct the solution path. Based on this statistical measure, we develop interpretable data augmentation methods that rely on distribution shifts or instance perturbations and validate that the performance of the learning-based solvers degenerates much on such augmented data. Moreover, fine-tuning learning-based solvers with augmented data further enhances their generalization abilities. In short, we decipher the limitations of learning-based TSP solvers tending to be overly greedy, which may have profound implications for AI-empowered combinatorial optimization solvers.

Figures (6)

• Figure 1: (a) Using random uniform Euclidean (RUE) data for model training may lead to algorithmic bias. In other words, the algorithm cannot guarantee that data from all structures are effective at the same scale. We can explicitly mine instances that cause the algorithm to fail using the nearest-neighbor density measure. (b) The mechanism behind the bias introduced by datasets. Specifically, the model is trained on a dataset generated from a uniform distribution, which has a high nearest-neighbor density characteristic. As a result, the model may incorrectly assume a causal relationship between nearest-neighbor density and the solution.
• Figure 2: (a) The preference of the model's solutions for nearest-neighbors at different node scales (50/100/200) generated from a uniform distribution. The green line represents the optimal solutions, while the red line represents the nearest-neighbor edges not covered by the optimal solutions. Most of the nearest-neighbor edges are included in the optimal solutions. (b) The statistical characteristics of nearest-neighbor density for instances with different node scales under the uniform distribution. As the node scale increases, the standard deviation decreases. (c) The model's performance on instances with different nearest-neighbor densities under uniform distribution. Generally, the model performs better (worse) on instances with high (low) nearest-neighbor density.
• Figure 3: (a) Illustration of a TSP instance containing 115,475 towns in the United States from the USA TSP Challenge. (b) A scale-free network with 100 nodes generated using the Barabási-Albert model (Appendix \ref{['Appendix-BA']}). (c) Distribution of nearest-neighbor distances for the TSP instances generated using Algorithm 1, exhibiting a multimodal distribution. (d) Distribution of nearest-neighbor distances for the traveling salesman problem instances generated using the uniform distribution. (e) Visualization of the instance with a size of 100 generated by Algorithm 1.
• Figure 4: (a) Visualization of the a280 instance from TSPLIB. (b) The optimal solution derived from the a280 instance, where the green line represents the optimal solution, the red line represents the nearest-neighbor edges not covered by the optimal solution, and the orange highlights the parallel pattern layout observed in this type of instance. (c) The artificial instances constructed based on the pattern in the drilling problem with ① representing the exact solution and ②-⑤ indicating the solutions of TSP-GNN joshi2022learning, TSP-Transformer Bresson2021, CycleFormer yook2024cycleformer, and the large-scale version of CycleFormer.
• Figure 5: (a) Distribution of the nearest-neighbor density $\rho_{50}$ for the augmented instances constructed by perturbing the drilling model instances. (b) Visualization of nine augmented instances with $\rho_{50}$ ranging from 0.1 to 0.9.

Theorems & Definitions (12)

• Definition 1: The Nearest-Neighbor Density
• Remark 1
• Theorem 1
• Remark 2
• Conjecture 2
• Definition 2: Language
• Definition 3: Generator
• Remark 3
• Theorem 3
• Definition 4: Efficient Algorithmic Coverage