From Black Box to Bijection: Interpreting Machine Learning to Build a Zeta Map Algorithm
Xiaoyu Huang, Blake Jackson, Kyu-Hwan Lee
TL;DR
The paper addresses the challenge of constructing explicit combinatorial bijections in algebraic combinatorics by training a transformer on Dyck-path data to learn the zeta map and then extracting a concrete algorithm from the model’s attention. The main approach demonstrates that a Minimal Dyck Transformer can reproduce the zeta map with perfect accuracy on moderate sizes and that cross-attention patterns reveal a level-driven mechanism akin to known sweep-type descriptions, culminating in the Scaffolding Map, a new sequential-output algorithm. The contributions include quantitative proof of concept for ML-driven bijection discovery, qualitative interpretability via attention analyses, and the introduction of Scaffolding Map as a provable combinatorial procedure derived from learned representations. This work suggests a general workflow for translating learned representations into explicit combinatorial algorithms, potentially accelerating discovery in algebraic combinatorics.
Abstract
There is a large class of problems in algebraic combinatorics which can be distilled into the same challenge: construct an explicit combinatorial bijection. Traditionally, researchers have solved challenges like these by visually inspecting the data for patterns, formulating conjectures, and then proving them. But what is to be done if patterns fail to emerge until the data grows beyond human scale? In this paper, we propose a new workflow for discovering combinatorial bijections via machine learning. As a proof of concept, we train a transformer on paired Dyck paths and use its learned attention patterns to derive a new algorithmic description of the zeta map, which we call the \textit{Scaffolding Map}.