Human Cognition Inspired RAG with Knowledge Graph for Complex Problem Solving

TL;DR

CogGRAG addresses the challenge of reliable, complex reasoning in KGQA by marrying top-down cognitive decomposition with graph-based retrieval and dual LLM verification. It constructs a tree-structured mind map to globally plan reasoning, performs one-pass structured retrieval of local and global KG knowledge, and executes bottom-up reasoning with a separate verifier that can re-think or abstain. Across HotpotQA, CWQ, WebQSP, and GRBENCH, CogGRAG consistently outperforms strong baselines and shows robustness across multiple backbone LMs. The approach demonstrates the value of explicit problem decomposition and self-verification for reliable, multi-hop reasoning in knowledge-rich tasks, enabling more scalable and trustworthy AI systems.

Abstract

Large Language Models (LLMs) have demonstrated significant potential across various domains. However, they often struggle with integrating external knowledge and performing complex reasoning, leading to hallucinations and unreliable outputs. Retrieval Augmented Generation (RAG) has emerged as a promising paradigm to mitigate these issues by incorporating external knowledge. Yet, conventional RAG approaches, especially those based on vector similarity, fail to effectively capture relational dependencies and support multi-step reasoning. In this work, we propose CogGRAG, a human cognition-inspired, graph-based RAG framework designed for Knowledge Graph Question Answering (KGQA). CogGRAG models the reasoning process as a tree-structured mind map that decomposes the original problem into interrelated subproblems and explicitly encodes their semantic relationships. This structure not only provides a global view to guide subsequent retrieval and reasoning but also enables self-consistent verification across reasoning paths. The framework operates in three stages: (1) top-down problem decomposition via mind map construction, (2) structured retrieval of both local and global knowledge from external Knowledge Graphs (KGs), and (3) bottom-up reasoning with dual-process self-verification. Unlike previous tree-based decomposition methods such as MindMap or Graph-CoT, CogGRAG unifies problem decomposition, knowledge retrieval, and reasoning under a single graph-structured cognitive framework, allowing early integration of relational knowledge and adaptive verification. Extensive experiments demonstrate that CogGRAG achieves superior accuracy and reliability compared to existing methods.

Figures (7)

• Figure 1: Representative workflow of two Retrieval-Augmented Generation paradigms for enhancing LLMs.
• Figure 2: The overall process of CogGRAG. Given a target question $Q$, CogGRAG first prompts the LLM to decompose it into a hierarchy of sub-problems in a top-down manner, constructing a structured mind map. Subsequently, CogGRAG prompts the LLM to extract both local level (entities and triples) and global level (subgraphs) key information from these questions. Finally, CogGRAG guides the LLM to perform bottom-up reasoning and verification based on the mind map and the retrieved knowledge, until the final answer is derived.
• Figure 3: Ablation study on the main components of CogGRAG.
• Figure 4: Case of CogGRAG.
• Figure 5: The prompt case of reasoning.