Int2Int: a framework for mathematics with transformers

TL;DR

Int2Int tackles the barrier to using transformer-based seq2seq models for mathematical discovery by treating problems and solutions as tokenized sequences and providing a complete, open-source PyTorch framework. The method centers on a modular encoder-decoder design with a projector for integers and related objects, enabling problem-agnostic experimentation via reusable components (model, optimizer, data loader, trainer, evaluator, tokenizer, data generator). The paper demonstrates practical use through two worked examples—the gcd of integers and predicting the rank of elliptic curves—emphasizing on-the-fly data generation or precomputed datasets and a thorough command-line workflow. It further contributes by detailing extension points (tokenizers, verifiers, generators) and visualization tools, aiming to lower the barrier for mathematicians to leverage transformers for discovery and verification inside established mathematical domains.

Abstract

This paper documents Int2Int, an open source code base for using transformers on problems of mathematical research, with a focus on number theory and other problems involving integers. Int2Int is a complete PyTorch implementation of a transformer architecture, together with training and evaluation loops, and classes and functions to represent, generate and decode common mathematical objects. Ancillary code for data preparation, and Jupyter Notebooks for visualizing experimental results are also provided. This document presents the main features of Int2Int, serves as its user manual, and provides guidelines on how to extend it. Int2Int is released under the MIT licence, at https://github.com/f-charton/Int2Int.