On Limitations of the Transformer Architecture
Binghui Peng, Srini Narayanan, Christos Papadimitriou
TL;DR
This paper reframes transformer hallucinations as a fundamental limitation in function composition, proving that a single $H$-headed self-attention layer cannot reliably compute $f(g(x))$ when $n\log n$ exceeds a bound set by $H(d+1)p$. Using communication complexity, it derives probabilistic error bounds and extends the analysis to iterated composition and Chain-of-Thought prompts, showing that CoT requires prompts of size at least $\Omega(\sqrt{N})$ to generalize. It further connects compositional tasks to log-space complexity, arguing that several elementary problems (circuit evaluation, derivability, certain SAT variants) are intractable for multi-layer Transformers unless major complexity class equalities hold (e.g., $\text{L}=\text{NL}$, etc.). The results collectively suggest fundamental, architecture-rooted barriers to robust compositionality in Transformers, with implications for model design and the interpretation of empirical failures. The work also discusses caveats, such as asymptotic nature and the potential for memory-time trading and architectural innovations to circumvent these limits, indicating avenues for future research.
Abstract
What are the root causes of hallucinations in large language models (LLMs)? We use Communication Complexity to prove that the Transformer layer is incapable of composing functions (e.g., identify a grandparent of a person in a genealogy) if the domains of the functions are large enough; we show through examples that this inability is already empirically present when the domains are quite small. We also point out that several mathematical tasks that are at the core of the so-called compositional tasks thought to be hard for LLMs are unlikely to be solvable by Transformers, for large enough instances and assuming that certain well accepted conjectures in the field of Computational Complexity are true.