Improving Rule-based Reasoning in LLMs using Neurosymbolic Representations

TL;DR

The paper tackles the difficulty of reliable rule-based numerical reasoning in large language models by introducing a neurosymbolic framework that encodes LLM hidden states into Vector Symbolic Algebras (VSAs). By performing symbolic computations in the VSA space and decoding the results back into the LLM, the approach achieves substantial improvements in numerical reasoning tasks with minimal overhead and selective intervention to avoid harming unseen tasks. Compared to chain-of-thought prompting and LoRA fine-tuning, the Neurosymbolic LLM delivers higher accuracy and lower cross-entropy loss across trained problem types, while preserving performance on out-of-distribution tasks via a gating mechanism. The method offers interpretability through explicit symbolic representations and demonstrates scalable potential for broader reasoning tasks that require precise rule-following.

Abstract

Large language models (LLMs) continue to face challenges in reliably solving reasoning tasks, particularly those that require precise rule following, as often found in mathematical reasoning. This paper introduces a novel neurosymbolic method that improves LLM reasoning by encoding hidden states into neurosymbolic vectors, enabling problem-solving within a neurosymbolic vector space. The results are decoded and merged with the original hidden state, significantly boosting the model's performance on numerical reasoning tasks. By offloading computation through neurosymbolic representations, this method enhances efficiency, reliability, and interpretability. Experimental results demonstrate an average of 88.6% lower cross-entropy loss and 15.4 times more problems correctly solved on a suite of mathematical reasoning tasks compared to chain-of-thought prompting and supervised fine-tuning (LoRA), without degrading performance on other tasks. We make our code available at: https://github.com/vdhanraj/Neurosymbolic-LLM.

Figures (6)

• Figure 1: A diagram of our method, showing how LLM hidden states are converted into compositional neurosymbolic representations. The encoder network converts the LLM hidden state to a neurosymbolic vector which can be queried to obtain the ones, tens, and hundreds digit of each number, as well as the type of problem being asked. This information is used by the neurosymbolic algorithm to find a solution to the problem, which the decoder converts from a neurosymbolic vector into an LLM hidden state vector, which is then added to the original LLM hidden state.
• Figure 2: Average RMSE loss of the encoder (blue) and decoder (red) across layers of the LLM.
• Figure 3: Classification Error Rate vs. Layer Number, across all problem types.
• Figure 4: Histogram of maximum similarity of queried problem type across all problem types, segregated per training and non-training problems
• Figure 5: Histogram of maximum problem type similarity for training problems vs. non-mathematical queries. None of the non-math queries exceed the 0.8 threshold.