Distortion Instead of Hallucination: The Effect of Reasoning Under Strict Constraints
Junichiro Niimi
TL;DR
This paper investigates how reasoning under strict journal-only constraints influences factual accuracy in LLM outputs for bibliographic recommendations. By comparing non-reasoning and reasoning-enabled models across two vendors (GPT-5.2 and Gemini 3 Flash) in a closed-system setting, it reveals a fundamental trade-off: non-reasoning models prioritize accuracy but violate constraints, while reasoning models reduce constraint violations at the cost of distorting known facts or fabricating entirely new records. The cross-model consistency of this pattern suggests a structural limitation of reasoning when external knowledge cannot be consulted. The work further shows that reasoning’s impact on overall authenticity is model-dependent, challenging the assumption that reasoning universally improves reliability and underscoring the need for careful constraint design and transparent reasoning in AI systems.
Abstract
With the widespread adoption of large language models (LLMs), hallucinations, which are non-factual fabrications in model outputs, have become serious concerns. Reasoning capabilities have received attention as a self-verification process to improve output reliability. However, the effect of reasoning within a closed system where LLMs cannot rely on external tools or knowledge has yet to be clarified. We therefore conduct experiments under strict constraints (recommending peer-reviewed journal articles in computer science) to examine the effect of reasoning across multiple models (GPT-5.2 and Gemini 3 Flash). Our results reveal a problematic trade-off between constraint compliance and factual accuracy. Non-reasoning models exhibit high constraint violation rates (66-75%) but maintain factual accuracy, while reasoning models reduce violations (13-26%) but systematically distort known facts to satisfy constraints and increase complete fabrication. This trade-off pattern is consistent across both models despite different architectures, indicating a fundamental limitation of reasoning. Furthermore, reasoning does not uniformly improve output authenticity: effects diverge by model, reflecting different allocations of the compliance-truthfulness trade-off. These findings challenge the assumption that reasoning universally improves reliability: reasoning models trade honest constraint violations for detection-resistant distortions.