Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mechanistic Interpretability for Neural TSP Solvers

Reuben Narad, Leonard Boussioux, Michael Wagner

TL;DR

This work addresses the interpretability gap of neural TSP solvers by applying sparse autoencoders to the encoder residual stream of a Transformer-based policy, revealing geometry-aligned internal features such as boundary detectors, cluster sensitivities, and separators. By training a strong RL-based TSP solver on 100-node Euclidean instances and analyzing the encoder with an overcomplete, sparse feature dictionary, the authors provide a model-internal account of what geometric structure the network computes before node selection. The study delivers a reproducible SAE workflow, a taxonomy of discovered feature types, and an interactive explorer to hypothesize about the solver's heuristics, paving the way for transparent hybrids that combine neural speed with algorithmic interpretability. The findings suggest that geometric structure emerges without explicit supervision and offer a foundation for debugging, robustness checks, and cross-layer circuit analysis in neural OR models.

Abstract

Neural networks have advanced combinatorial optimization, with Transformer-based solvers achieving near-optimal solutions on the Traveling Salesman Problem (TSP) in milliseconds. However, these models operate as black boxes, providing no insight into the geometric patterns they learn or the heuristics they employ during tour construction. We address this opacity by applying sparse autoencoders (SAEs), a mechanistic interpretability technique, to a Transformer-based TSP solver, representing the first application of activation-based interpretability methods to operations research models. We train a pointer network with reinforcement learning on 100-node instances, then fit an SAE to the encoder's residual stream to discover an overcomplete dictionary of interpretable features. Our analysis reveals that the solver naturally develops features mirroring fundamental TSP concepts: boundary detectors that activate on convex-hull nodes, cluster-sensitive features responding to locally dense regions, and separator features encoding geometric partitions. These findings provide the first model-internal account of what neural TSP solvers compute before node selection, demonstrate that geometric structure emerges without explicit supervision, and suggest pathways toward transparent hybrid systems that combine neural efficiency with algorithmic interpretability. Interactive feature explorer: https://reubennarad.github.io/TSP_interp

Mechanistic Interpretability for Neural TSP Solvers

TL;DR

This work addresses the interpretability gap of neural TSP solvers by applying sparse autoencoders to the encoder residual stream of a Transformer-based policy, revealing geometry-aligned internal features such as boundary detectors, cluster sensitivities, and separators. By training a strong RL-based TSP solver on 100-node Euclidean instances and analyzing the encoder with an overcomplete, sparse feature dictionary, the authors provide a model-internal account of what geometric structure the network computes before node selection. The study delivers a reproducible SAE workflow, a taxonomy of discovered feature types, and an interactive explorer to hypothesize about the solver's heuristics, paving the way for transparent hybrids that combine neural speed with algorithmic interpretability. The findings suggest that geometric structure emerges without explicit supervision and offer a foundation for debugging, robustness checks, and cross-layer circuit analysis in neural OR models.

Abstract

Neural networks have advanced combinatorial optimization, with Transformer-based solvers achieving near-optimal solutions on the Traveling Salesman Problem (TSP) in milliseconds. However, these models operate as black boxes, providing no insight into the geometric patterns they learn or the heuristics they employ during tour construction. We address this opacity by applying sparse autoencoders (SAEs), a mechanistic interpretability technique, to a Transformer-based TSP solver, representing the first application of activation-based interpretability methods to operations research models. We train a pointer network with reinforcement learning on 100-node instances, then fit an SAE to the encoder's residual stream to discover an overcomplete dictionary of interpretable features. Our analysis reveals that the solver naturally develops features mirroring fundamental TSP concepts: boundary detectors that activate on convex-hull nodes, cluster-sensitive features responding to locally dense regions, and separator features encoding geometric partitions. These findings provide the first model-internal account of what neural TSP solvers compute before node selection, demonstrate that geometric structure emerges without explicit supervision, and suggest pathways toward transparent hybrid systems that combine neural efficiency with algorithmic interpretability. Interactive feature explorer: https://reubennarad.github.io/TSP_interp
Paper Structure (19 sections, 4 equations, 4 figures)

This paper contains 19 sections, 4 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Classical heuristics.
  4. Learning-based approaches to the TSP.
  5. Interpretability and mechanistic interpretability.
  6. Training a Neural TSP Solver
  7. Neural Network Architecture
  8. Training with RL
  9. Interpretability with Sparse Autoencoders
  10. Interpretability Goal
  11. SAE Background
  12. SAE Training Details
  13. Feature Analysis
  14. Feature Activation Mechanics
  15. Visualizing a Feature
  16. ...and 4 more sections

Figures (4)

  • Figure 1: System architecture of the TSP solver transformer model, adopting the architecture from Kool2019.
  • Figure 2: Example TSP instances drawn from different distributions.
  • Figure 3: The SAE learns an overcomplete and sparse representation of neuron activations of the TSP solver model
  • Figure 4: Each panel is a visualization of a single SAE feature. It overlays nodes from 10 TSP instances, with the marker shape indicating the instance this node was drawn from. Color shows SAE activation, with purple = low and yellow = high.