Neural Isomorphic Fields: A Transformer-based Algebraic Numerical Embedding

TL;DR

Neural Isomorphic Fields (NIF) establish a transformer-based approach to embed rational numbers as fixed-length vectors while preserving key algebraic properties. Numbers are encoded as digit sequences and mapped to embeddings that aim to form an ordered field isomorphic to $Q$, with an Abelian Neural Operator and neural order enforcing addition, multiplication, and ordering constraints through dedicated losses. Empirical results show addition properties are robust (≈95–99% accuracy on core tests) while multiplication remains more challenging (≈53–73% across tests), highlighting both promise and areas for improvement. The work offers a path toward algebraically faithful number embeddings with potential impact on domains requiring precise arithmetic reasoning in neural models.

Abstract

Neural network models often face challenges when processing very small or very large numbers due to issues such as overflow, underflow, and unstable output variations. To mitigate these problems, we propose using embedding vectors for numbers instead of directly using their raw values. These embeddings aim to retain essential algebraic properties while preventing numerical instabilities. In this paper, we introduce, for the first time, a fixed-length number embedding vector that preserves algebraic operations, including addition, multiplication, and comparison, within the field of rational numbers. We propose a novel Neural Isomorphic Field, a neural abstraction of algebraic structures such as groups and fields. The elements of this neural field are embedding vectors that maintain algebraic structure during computations. Our experiments demonstrate that addition performs exceptionally well, achieving over 95 percent accuracy on key algebraic tests such as identity, closure, and associativity. In contrast, multiplication exhibits challenges, with accuracy ranging from 53 percent to 73 percent across various algebraic properties. These findings highlight the model's strengths in preserving algebraic properties under addition while identifying avenues for further improvement in handling multiplication.

Figures (8)

• Figure 1: The NIF - model architecture.
• Figure 2: Stacked Abelian Neural Operator.
• Figure 3: Overall training procedure for creating Neural Isomorphic Field.
• Figure 4: A portion of the tested distributions for the length of numbers
• Figure 5: Learning rate schedule of Adam optimizer

Theorems & Definitions (23)

• Definition 3.1
• Proposition 3.1
• Theorem 3.1
• Theorem 3.2
• Theorem 3.3
• Theorem 3.4
• Lemma A.1
• proof
• Theorem A.2
• proof