From Chains to Graphs: Self-Structured Reasoning for General-Domain LLMs

TL;DR

The paper tackles inconsistent reasoning in open-domain QA by introducing Self-Graph Reasoning (SGR), a framework that externalizes latent reasoning into explicit graph structures before answering. It builds a 10K-graph supervision dataset from LogiQA and trains LLaMA-3.3-70B with LoRA to generate structured reasoning graphs, achieving a 61.03% average accuracy across five benchmarks and competitive performance with GPT-4o while outperforming several open-source baselines. SGR demonstrates strong cross-domain generalization, including MedQA and MathQA, and highlights the benefits of graph-structured reasoning over linear CoT. The work also provides ablations, a case study, and a cost analysis, underscoring scalability considerations and potential for larger models to yield further gains.

Abstract

Large Language Models (LLMs) show strong reasoning ability in open-domain question answering, yet their reasoning processes are typically linear and often logically inconsistent. In contrast, real-world reasoning requires integrating multiple premises and solving subproblems in parallel. Existing methods, such as Chain-of-Thought (CoT), express reasoning in a linear textual form, which may appear coherent but frequently leads to inconsistent conclusions. Recent approaches rely on externally provided graphs and do not explore how LLMs can construct and use their own graph-structured reasoning, particularly in open-domain QA. To fill this gap, we novelly explore graph-structured reasoning of LLMs in general-domain question answering. We propose Self-Graph Reasoning (SGR), a framework that enables LLMs to explicitly represent their reasoning process as a structured graph before producing the final answer. We further construct a graph-structured reasoning dataset that merges multiple candidate reasoning graphs into refined graph structures for model training. Experiments on five QA benchmarks across both general and specialized domains show that SGR consistently improves reasoning consistency and yields a 17.74% gain over the base model. The LLaMA-3.3-70B model fine-tuned with SGR performs comparably to GPT-4o and surpasses Claude-3.5-Haiku, demonstrating the effectiveness of graph-structured reasoning.

Figures (11)

• Figure 1: Comparison of reasoning and answering paradigms in general-domain question answering. (a) Direct Answering without explicit reasoning. (b) Linear reasoning with weak logical alignment between intermediate reasoning and the final answer. (c) Our Graph-structured reasoning with explicit logical connections, yielding higher reasoning consistency.
• Figure 2: Overview of the proposed Self-Graph Reasoning (SGR).
• Figure 3: An illustrative example of the Self-Graph Reasoning framework. Our method constructs a structured reasoning graph where each node is explicitly grounded in its predecessors to ensure a logically consistent and clear path to the final answer. For comparison, we also illustrate other paradigms: Direct Answering lacks an explicit thinking process and is prone to errors, while Linear Reasoning often exhibits process-answer inconsistency (e.g., reaching a correct answer through flawed factual steps).
• Figure 4: Comparison of accuracy under different reasoning paradigms on the AIW dataset. We evaluate three paradigms: Direct Answering (DA), Linear Reasoning (LR), and our Self-Graph Reasoning (SGR) across GPT-4o, Claude-3.5-haiku, and LLaMA-3.3-70B.
• Figure 5: Comparison of accuracy between the base models and our SGR across all five benchmarks for LLaMA-3.3-8B and LLaMA-3.3-70B.