Enhancing Large Language Model Efficiencyvia Symbolic Compression: A Formal Approach Towards Interpretability

TL;DR

Addresses token efficiency in LLMs for code generation and reasoning, where token redundancy inflates inference costs and harms interpretability. Proposes a formal symbolic compression framework that blends Kolmogorov-based symbolic density, SKI combinator encoding, context-aware inference, a differentiable compressor, MDL-based symbolic layers, and PEFT with the GAEL language to achieve efficiency and interpretability. Key contributions include a quantitative link between symbolic density and interpretability with $\rho = \frac{\mathcal{K}(s)}{|s|}$, a differentiable compression factor, recursive SKI encoding with a theoretical $O(\log n / n)$ bound, dynamic balancing for context inference, and PEFT-enabled GAEL deployment, achieving a token compression rate of 78.3% and up to 62% gains in logical traceability on standard benchmarks. Significance: reduces token usage and inference cost while enhancing structural explicitness and traceability, enabling more transparent and scalable symbolic reasoning in LLMs.

Abstract

Large language models (LLMs) face significant token efficiency bottlenecks in code generation and logical reasoning tasks, a challenge that directly impacts inference cost and model interpretability. This paper proposes a formal framework based on symbolic compression,integrating combinatory logic, information-theoretic optimal encoding, and context-aware inference techniques to achieve a step-change improvement in token efficiency while preserving semantic integrity. We establish a mathematical framework within a functional programming paradigm, derive the quantitative relationship between symbolic density and model interpretability, and propose a differentiable compression factor metric to evaluate encoding efficiency. Furthermore, we leverage parameter-efficient fine-tuning (PEFT) techniques to achieve a low-cost application of the GAEL language. Experimental results show that this method achieves a 78.3% token compression rate in code generation tasks while improving logical traceability by 62% through structural explicitness. This research provides new theoretical tools for efficient inference in LLMs and opens a symbolic path for modelinterpretability research.