Memorization, Emergence, and Explaining Reversal Failures: A Controlled Study of Relational Semantics in LLMs

TL;DR

This work investigates whether autoregressive LLMs truly internalize relational semantics (e.g., symmetry and inversion) or rely on superficial co-occurrence patterns. A fully controlled KG-based synthetic framework is used to train GPT-style models from scratch, with systematic evaluation across Memorize QA, Logic QA, and in-context learning to unseen entities; findings reveal a sharp emergence of relational semantics under sufficient logic-bearing supervision, even in shallow models, and a strong link between generalized performance and stable intermediate-layer representations. The study shows reversal-type failures are predominantly driven by left-to-right autoregressive order bias rather than missing inversion semantics, with bidirectional training mitigating the effect; diffusion-based models display reduced sensitivity to order bias. These insights advance understanding of how relational reasoning can emerge in LLMs under controlled conditions and suggest practical training and evaluation strategies to encourage robust relational inference in language models.

Abstract

Autoregressive LLMs perform well on relational tasks that require linking entities via relational words (e.g., father/son, friend), but it is unclear whether they learn the logical semantics of such relations (e.g., symmetry and inversion logic) and, if so, whether reversal-type failures arise from missing relational semantics or left-to-right order bias. We propose a controlled Knowledge Graph-based synthetic framework that generates text from symmetric/inverse triples, train GPT-style autoregressive models from scratch, and evaluate memorization, logical inference, and in-context generalization to unseen entities to address these questions. We find a sharp phase transition in which relational semantics emerge with sufficient logic-bearing supervision, even in shallow (2-3 layer) models, and that successful generalization aligns with stable intermediate-layer signals. Finally, order-matched forward/reverse tests and a diffusion baseline indicate that reversal failures are primarily driven by autoregressive order bias rather than deficient inversion semantics.

Figures (13)

• Figure 1: Two-stage inquiry into relational-word understanding in autoregressive language models. (a) Models can memorize symmetric and inverse relational facts and perform forward-order logic QA; both completion- and QA-based in-context evaluations on unseen entities exhibit a sharp phase transition with sufficient logic-bearing supervision, indicating the emergence of relational semantics. (b) While inverse relations perform comparably to their original under forward queries, reversed-order queries show a substantial performance drop, revealing the reversal curse of order sensitivity.
• Figure 2: Overview of the KG-synthetic data generation framework, model training process, and evaluation detail.
• Figure 3: Memorize QA ($\text{Q}_{\text{Mem}}$) evaluation under varying training epochs $E$, template size $K$, training sample size $N$, and model depth $L$. (a) Vary $E$ with $K,L,N$ fixed. (b) Vary $K$ with $N,L$ fixed. (c) Vary $N$ with $K,L$ fixed. (d) Vary $L$ with $K,N$ fixed.
• Figure 4: Logic and ICL evaluation. (a) Vary $E$ with $K,L,N$ fixed. (b) Vary $K$ with $N,L$ fixed.
• Figure 5: Effect of model depth on logic and ICL completion evaluations. (a) In-context performance as $L$ varies under fixed $K$ and $N$. (b--d) For shallow models ($L=1,2,3$), evaluation performance across $N$ with fixed $K$.