Mitigating Prompt-Induced Hallucinations in Large Language Models via Structured Reasoning

TL;DR

This work tackles prompt-induced hallucinations in large language models by introducing a code-guided, externally grounded knowledge-distillation chain-style framework. By integrating structured knowledge graphs and a code module into the reasoning pipeline, the approach constrains intermediate steps, enabling verifiable, low-hallucination multi-step inference. Empirical results on GPT-4 and LLaMA 3.3 across diverse datasets show consistent HIT@K gains, with several settings surpassing 95% and notable improvements over baselines. The method enhances robustness, generalization, and interpretability, with potential extensions to multimodal reasoning and retrieval-augmented strategies for broader applicability.

Abstract

To address hallucination issues in large language models (LLMs), this paper proposes a method for mitigating prompt-induced hallucinations. Building on a knowledge distillation chain-style model, we introduce a code module to guide knowledge-graph exploration and incorporate code as part of the chain-of-thought prompt, forming an external knowledge input that provides more accurate and structured information to the model. Based on this design, we develop an improved knowledge distillation chain-style model and leverage it to analyze and constrain the reasoning process of LLMs, thereby improving inference accuracy. We empirically evaluate the proposed approach using GPT-4 and LLaMA-3.3 on multiple public datasets. Experimental results demonstrate that incorporating code modules significantly enhances the model's ability to capture contextual information and effectively mitigates prompt-induced hallucinations. Specifically, HIT@1, HIT@3, and HIT@5 improve by 15.64%, 13.38%, and 13.28%, respectively. Moreover, the proposed method achieves HIT@1, HIT@3, and HIT@5 scores exceeding 95% across several evaluation settings. These results indicate that the proposed approach substantially reduces hallucination behavior while improving the accuracy and verifiability of large language models.

Figures (8)

• Figure 1: Structure of the knowledge distillation chain model.
• Figure 2: The process of suggesting hallucination problem-solving methods based on the large model based on the improved knowledge distillation chain.
• Figure 3: Simulation experiments.
• Figure 4: Verification results of the improvement of the knowledge distillation chain model.
• Figure 5: Robustness Verification Results.