Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beware of the Classical Benchmark Instances for the Traveling Salesman Problem with Time Windows

Francisco J. Soulignac

TL;DR

This work challenges the continued use of classical TSPTW benchmarks for evaluating TSPTW-M and TSPTW-D by demonstrating that a simple exact informed search can solve large instances extremely quickly. The authors introduce a backward best-first search solver with an Unreachable Function pruning mechanism, and extend the approach to the TSPTW-D via a sliding-window and reverse-network technique. Their experimental results show that 50+ customer benchmarks are easily solved, highlighting biases in benchmark design and the potential misrepresentation of algorithmic performance. The paper advocates for harder, varied time-window generation (e.g., beta-based widening) and cautions against relying on these benchmarks for benchmarking and training data generation in machine learning contexts.

Abstract

We propose a simple and exact informed search method for the Traveling Salesman Problem with Time Windows and Makespan objective (TSPTW-M) that solves all instances of the classical benchmark with 50 or more customers in less than ten seconds each. Applying this algorithm as an off-the-shelf method, we also solve all but one of these instances for the Duration objective. Our main conclusion is that these instances should no longer be employed for evaluating the TSPTW-M and its Duration variant: they can be ``hacked'' to yield results that seem outstanding at first sight. Additionally, caution is advised when designing hard training sets for machine learning algorithms.

Beware of the Classical Benchmark Instances for the Traveling Salesman Problem with Time Windows

TL;DR

This work challenges the continued use of classical TSPTW benchmarks for evaluating TSPTW-M and TSPTW-D by demonstrating that a simple exact informed search can solve large instances extremely quickly. The authors introduce a backward best-first search solver with an Unreachable Function pruning mechanism, and extend the approach to the TSPTW-D via a sliding-window and reverse-network technique. Their experimental results show that 50+ customer benchmarks are easily solved, highlighting biases in benchmark design and the potential misrepresentation of algorithmic performance. The paper advocates for harder, varied time-window generation (e.g., beta-based widening) and cautions against relying on these benchmarks for benchmarking and training data generation in machine learning contexts.

Abstract

We propose a simple and exact informed search method for the Traveling Salesman Problem with Time Windows and Makespan objective (TSPTW-M) that solves all instances of the classical benchmark with 50 or more customers in less than ten seconds each. Applying this algorithm as an off-the-shelf method, we also solve all but one of these instances for the Duration objective. Our main conclusion is that these instances should no longer be employed for evaluating the TSPTW-M and its Duration variant: they can be ``hacked'' to yield results that seem outstanding at first sight. Additionally, caution is advised when designing hard training sets for machine learning algorithms.

Paper Structure

This paper contains 12 sections, 1 theorem, 4 equations, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Our contributions
  3. Problem Statement
  4. Benchmark instances
  5. A Simple Solver for the TSPTW-M
  6. Implementation details.
  7. The Unreachable Function
  8. Solving the integer TSPTW-D
  9. Computational Results
  10. Results for the TSPTW-M
  11. Results for the TSPTW-D
  12. Conclusions

Key Result

Proposition 1

Let $R$ be a partial route from vertex $v$ to the end depot $n+1$ with $\delta^{-1}_{\rm m}(R) \geq t_0$. If there exists a partial route $Q \neq R$ from $v$ to $n+1$ such that its earliest arrival time is $\delta_{\rm m}(Q) < ub$, it visits the vertices $V(Q) = V(R)$, its latest departure is $\delt then $R$ is $ub$-dominated by $Q$.

Theorems & Definitions (1)

  • Proposition 1