Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers

TL;DR

Pretrained language models encode numeric information with a highly structured, often sinusoidal, embedding pattern. The authors introduce sin-based and Fourier-inspired probes that decode integers from number embeddings with near-perfect accuracy across multiple open-source LMs, revealing that exact numeric representations survive pretraining. They show that the precision of these numeric embeddings explains a substantial portion of elementary arithmetic errors and that aligning embeddings to the discovered pattern can reduce such errors. This work challenges the notion that LMs cannot reliably represent exact quantities and demonstrates that targeted embedding interventions can improve numeracy without external calculation tools.

Abstract

Pretrained language models (LMs) are prone to arithmetic errors. Existing work showed limited success in probing numeric values from models' representations, indicating that these errors can be attributed to the inherent unreliability of distributionally learned embeddings in representing exact quantities. However, we observe that previous probing methods are inadequate for the emergent structure of learned number embeddings with sinusoidal patterns. In response, we propose a novel probing technique that decodes numeric values from input embeddings with near-perfect accuracy across a range of open-source LMs. This proves that after the sole pre-training, LMs represent numbers with remarkable precision. Finally, we find that the embeddings' precision, judged by our probe's accuracy, explains a large portion of LM's errors in elementary arithmetic, and show that aligning the embeddings with the pattern our probes discover can mitigate these errors.

Figures (6)

• Figure 1: Overview of probes' accuracy ($\uparrow$).
• Figure 2: Visualization of PCA (DIM=16) reduced number embeddings, selected models. Although most model exhibit relatively regular wave-like patterns, OLMO 2 32B exhibit little regularity.
• Figure 3: Maximal contribution (magnitude) of each Fourier base frequency's to embedding features in PCA (d=128) reduced space. Sparsity in this plot indicates strong regularity in the hidden structure of model embeddings. OLMo 2 32B has noticeably stronger contribution of all low-contribution frequencies, indicating high irregularity.
• Figure 4: Probability mass on the predicted output token when the LLM yields a correct vs. incorrect answer.
• Figure 5: Head activations across arithmetic tasks for Llama 3 1B, broken down by task (addition and subtraction) and success (correct or incorrect computation.