Computational Algebra with Attention: Transformer Oracles for Border Basis Algorithms

TL;DR

The paper tackles solving zero-dimensional polynomial systems by augmenting border-basis computations with a Transformer-based oracle that identifies and prunes expensive reductions while guaranteeing correct results. It introduces the Oracle Border Basis Algorithm (OBBA), a data-efficient supervised learning framework that learns to select a small subset of reductions during the final, most time-consuming stage of computation. The authors develop a monomial-centered input representation, border-basis sampling via a new sampling theorem, and an ideal-invariant generator transform to create diverse training data. Empirically, OBBA achieves up to $3.5\times$ speedups over the traditional approach, with solid performance in both in-distribution and out-of-distribution settings, demonstrating a practical integration of learning into symbolic computation.

Abstract

Solving systems of polynomial equations, particularly those with finitely many solutions, is a crucial challenge across many scientific fields. Traditional methods like Gröbner and Border bases are fundamental but suffer from high computational costs, which have motivated recent Deep Learning approaches to improve efficiency, albeit at the expense of output correctness. In this work, we introduce the Oracle Border Basis Algorithm, the first Deep Learning approach that accelerates Border basis computation while maintaining output guarantees. To this end, we design and train a Transformer-based oracle that identifies and eliminates computationally expensive reduction steps, which we find to dominate the algorithm's runtime. By selectively invoking this oracle during critical phases of computation, we achieve substantial speedup factors of up to 3.5x compared to the base algorithm, without compromising the correctness of results. To generate the training data, we develop a sampling method and provide the first sampling theorem for border bases. We construct a tokenization and embedding scheme tailored to monomial-centered algebraic computations, resulting in a compact and expressive input representation, which reduces the number of tokens to encode an $n$-variate polynomial by a factor of $O(n)$. Our learning approach is data efficient, stable, and a practical enhancement to traditional computer algebra algorithms and symbolic computation.

Figures (13)

• Figure 1: Border basis concepts: (a) A border basis with order ideal $\{1,y\}$ and border terms $\{y^2,xy,x\}$. (b) BBA's iterative expansion of $\mathcal{V}$, showing leading terms: two initial polynomials yield four expansions, then eight more - though only two out of twelve were necessary. (c) The oracle approach achieves the same result with just four targeted expansions.
• Figure 2: (${\mathbb{F}}_{31}$, $k=5$). The term truncation and monomial embedding significantly reduce input size. See also \ref{['fig:input-sequence-compression']}.
• Figure 3: Speed-up of OBBA over IBBA on OOD systems with $n=4$ variables and increased degree. Each point averages 100 random instances per field. The relative border-gap $\frac{|\mathcal{V}|}{|\mathcal{L}|}$ is the threshold that decides when the oracle is invoked; a ratio of 1 corresponds to IBBA, where the oracle is never used. Although the oracle is trained only on systems of total degree $2$ for $n=4$, it generalizes to degrees $3, 4, 6,$ and $8$, achieving up to $1.8\times$ speed-up even for degree $8$. The average runtime for IBBA on the OOD systems is two orders of magnitude higher than for the in distribution case.
• Figure 4: The gallery of randomly sampled order ideals for $n=2$ and $\bm{d} = (10, 10)$.
• Figure 5: The empirical success rate of the backward transform from $G$ to $F$ without changing ideals. As Theorem \ref{['thm:backward-transform']} and Corollary \ref{['cor:backward-transform']} suggest, the success rate is zero for $|F| = n$ and close to one for $|F| > n$, and larger field order and number of variables increase the success rate.

Theorems & Definitions (47)

• Definition 2.1
• Definition 2.2
• Theorem 3.1
• Definition 4.1
• Theorem 4.2
• Remark 4.3
• Remark 4.4
• Theorem 4.5
• Definition 4.5
• Theorem A.1