Can LLMs Compute with Reasons?

TL;DR

Can LLMs Compute with Reasons? addresses the challenge of mathematical reasoning in LLMs and smaller LangSLMs by proposing an Inductive Learning framework that coordinates a pair of LLMs—a general-purpose GP and an equation-focused EQ—through a distributed network and cross-inference. The approach uses error-based hints and voting across multiple model pairs to refine algebraic and logical reasoning, achieving a main output of 50.29% on GSM8K benchmarks and outperforming relevant baselines like Phi1.5 while remaining competitive with TinyGSM in terms of logic transfer. The work demonstrates a practical pathway for transferring robust, logic-based reasoning capabilities to smaller models and suggests directions for reinforcement learning-driven tuning to further enhance reasoning. These findings have potential impact across domains requiring reliable mathematical reasoning in language-based systems.

Abstract

Large language models (LLMs) often struggle with complex mathematical tasks, prone to "hallucinating" incorrect answers due to their reliance on statistical patterns. This limitation is further amplified in average Small LangSLMs with limited context and training data. To address this challenge, we propose an "Inductive Learning" approach utilizing a distributed network of SLMs. This network leverages error-based learning and hint incorporation to refine the reasoning capabilities of SLMs. Our goal is to provide a framework that empowers SLMs to approach the level of logic-based applications achieved by high-parameter models, potentially benefiting any language model. Ultimately, this novel concept paves the way for bridging the logical gap between humans and LLMs across various fields.

Figures (3)

• Figure 1: Network Topology
• Figure 2: Distributed computation
• Figure 3: The cross inference for increasing probability