Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Algorithmic Language Models with Neurally Compiled Libraries

Lucas Saldyt, Subbarao Kambhampati

TL;DR

The paper tackles the problem that large language models struggle with true algorithmic reasoning due to optimization and architectural limitations. It proposes neural compilation of a differentiable library of fundamental operations into a transformer (LLaMA3.2) to enable native execution of algorithms with adaptive depth, memory, and a differentiable interpreter. Key contributions include a detailed differentiable memory/register machine architecture, a call-based program library, and experiments showing programmable arithmetic and limited sorting in small-scale models, along with insights on tokenization and representation challenges. The results indicate that differentiable computers can bootstrap algorithmic reasoning in LLMs and point to future work on parallel differentiable hardware and more robust compositionality for broader, reliable reasoning capabilities.

Abstract

Important tasks such as reasoning and planning are fundamentally algorithmic, meaning that solving them robustly requires acquiring true reasoning or planning algorithms, rather than shortcuts. Large Language Models lack true algorithmic ability primarily because of the limitations of neural network optimization algorithms, their optimization data and optimization objective, but also due to architectural inexpressivity. To solve this, our paper proposes augmenting LLMs with a library of fundamental operations and sophisticated differentiable programs, so that common algorithms do not need to be learned from scratch. We add memory, registers, basic operations, and adaptive recurrence to a transformer architecture built on LLaMA3. Then, we define a method for directly compiling algorithms into a differentiable starting library, which is used natively and propagates gradients for optimization. In this preliminary study, we explore the feasability of augmenting LLaMA3 with a differentiable computer, for instance by fine-tuning small transformers on simple algorithmic tasks with variable computational depth.

Algorithmic Language Models with Neurally Compiled Libraries

TL;DR

The paper tackles the problem that large language models struggle with true algorithmic reasoning due to optimization and architectural limitations. It proposes neural compilation of a differentiable library of fundamental operations into a transformer (LLaMA3.2) to enable native execution of algorithms with adaptive depth, memory, and a differentiable interpreter. Key contributions include a detailed differentiable memory/register machine architecture, a call-based program library, and experiments showing programmable arithmetic and limited sorting in small-scale models, along with insights on tokenization and representation challenges. The results indicate that differentiable computers can bootstrap algorithmic reasoning in LLMs and point to future work on parallel differentiable hardware and more robust compositionality for broader, reliable reasoning capabilities.

Abstract

Important tasks such as reasoning and planning are fundamentally algorithmic, meaning that solving them robustly requires acquiring true reasoning or planning algorithms, rather than shortcuts. Large Language Models lack true algorithmic ability primarily because of the limitations of neural network optimization algorithms, their optimization data and optimization objective, but also due to architectural inexpressivity. To solve this, our paper proposes augmenting LLMs with a library of fundamental operations and sophisticated differentiable programs, so that common algorithms do not need to be learned from scratch. We add memory, registers, basic operations, and adaptive recurrence to a transformer architecture built on LLaMA3. Then, we define a method for directly compiling algorithms into a differentiable starting library, which is used natively and propagates gradients for optimization. In this preliminary study, we explore the feasability of augmenting LLaMA3 with a differentiable computer, for instance by fine-tuning small transformers on simple algorithmic tasks with variable computational depth.
Paper Structure (40 sections, 9 equations, 4 figures, 2 tables)

This paper contains 40 sections, 9 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Neural Compilation
  3. Related Work
  4. Previous Neural Compilation Techniques
  5. Differentiable Computing and Program Synthesis
  6. Large Language Model Tool Use
  7. Gradient Estimation
  8. Augmenting LLaMA 3.2 with a Differentiable Library
  9. Library Structure
  10. Parsing/Selection Problem
  11. Call Instruction
  12. Model Background
  13. Differentiable Memory
  14. Differentiable Registers
  15. Differentiable Register Machine
  16. ...and 25 more sections

Figures (4)

  • Figure 1: The proposed architecture, which adds a library of differentiable programs to LLaMA3. In principle, these programs can be composed to flexibly complete new tasks without re-learning individual subprograms. For example, the model may first parse text, sort a list, and then count the characters in the first title by composing sort() and count() with the LLM's parsing ability.
  • Figure 2: Differentiable Register Machine, Introduced in Bunel 2016 bunel2016adaptive
  • Figure 3: Circuits vs Tables
  • Figure 4: Insertion Sort