Improving the Reliability of LLMs: Combining CoT, RAG, Self-Consistency, and Self-Verification
Adarsh Kumar, Hwiyoon Kim, Jawahar Sai Nathani, Neil Roy
TL;DR
This paper tackles hallucination in large language models by integrating grounding and verification strategies. It combines Chain-of-Thought prompting with Retrieval-Augmented Generation, along with Self-Consistency and Self-Verification, and evaluates these approaches across three benchmark datasets and multiple model families. Key findings show substantial reductions in hallucinations and improvements in factual accuracy, with Self-Verification often delivering the strongest gains. The work has practical implications for building more reliable LLMs in open-ended tasks and provides directions for scalable grounding and verification.
Abstract
Hallucination, where large language models (LLMs) generate confident but incorrect or irrelevant information, remains a key limitation in their application to complex, open-ended tasks. Chain-of-thought (CoT) prompting has emerged as a promising method for improving multistep reasoning by guiding models through intermediate steps. However, CoT alone does not fully address the hallucination problem. In this work, we investigate how combining CoT with retrieval-augmented generation (RAG), as well as applying self-consistency and self-verification strategies, can reduce hallucinations and improve factual accuracy. By incorporating external knowledge sources during reasoning and enabling models to verify or revise their own outputs, we aim to generate more accurate and coherent responses. We present a comparative evaluation of baseline LLMs against CoT, CoT+RAG, self-consistency, and self-verification techniques. Our results highlight the effectiveness of each method and identify the most robust approach for minimizing hallucinations while preserving fluency and reasoning depth.