Geometric Generality of Transformer-Based Gröbner Basis Computation
Yuta Kambe, Yota Maeda, Tristan Vaccon
TL;DR
The paper addresses training Transformer-based methods to compute Gröbner bases by ensuring the training data are generically representative of the problem space. It develops a rigorous geometric framework around Zariski density to guarantee that generated datasets sample the space of Gröner bases broadly, linking algebraic density to practical generalization. The main contributions are a generalized dataset-construction algorithm producing $F=AG$ with $\langle F\rangle = \langle G\rangle$, and a density proof showing $\mathcal{F}_0$ is dense in the target space under mild conditions (e.g., $m \ge 2n \ge 3$, irreducible $\mathcal{X}_{\le D}$, and Hilbertian $K$). These results connect rigorous algebraic geometry with dataset design, offering a theoretical justification for training on diverse inputs to accelerate Gröbner-basis computation using Transformers and clarifying the role of the coefficient field in learning performance.
Abstract
The intersection of deep learning and symbolic mathematics has seen rapid progress in recent years, exemplified by the work of Lample and Charton. They demonstrated that effective training of machine learning models for solving mathematical problems critically depends on high-quality, domain-specific datasets. In this paper, we address the computation of Gröbner basis using Transformers. While a dataset generation method tailored to Transformer-based Gröbner basis computation has previously been proposed, it lacked theoretical guarantees regarding the generality or quality of the generated datasets. In this work, we prove that datasets generated by the previously proposed algorithm are sufficiently general, enabling one to ensure that Transformers can learn a sufficiently diverse range of Gröbner bases. Moreover, we propose an extended and generalized algorithm to systematically construct datasets of ideal generators, further enhancing the training effectiveness of Transformer. Our results provide a rigorous geometric foundation for Transformers to address a mathematical problem, which is an answer to Lample and Charton's idea of training on diverse or representative inputs.