Leveraging LLMs for Hypothetical Deduction in Logical Inference: A Neuro-Symbolic Approach

TL;DR

By enabling an LLM to autonomously perform the transition from propositional logic extraction to sophisticated logical reasoning, LINA not only bolsters the resilience of the reasoning process but also eliminates the dependency on external solvers.

Abstract

Large Language Models (LLMs) have exhibited remarkable potential across a wide array of reasoning tasks, including logical reasoning. Although massive efforts have been made to empower the logical reasoning ability of LLMs via external logical symbolic solvers, crucial challenges of the poor generalization ability to questions with different features and inevitable question information loss of symbolic solver-driven approaches remain unresolved. To mitigate these issues, we introduce LINA, a LLM-driven neuro-symbolic approach for faithful logical reasoning. By enabling an LLM to autonomously perform the transition from propositional logic extraction to sophisticated logical reasoning, LINA not only bolsters the resilience of the reasoning process but also eliminates the dependency on external solvers. Additionally, through its adoption of a hypothetical-deductive reasoning paradigm, LINA effectively circumvents the expansive search space challenge that plagues traditional forward reasoning methods. Empirical evaluations demonstrate that LINA substantially outperforms both established propositional logic frameworks and conventional prompting techniques across a spectrum of five logical reasoning tasks. Specifically, LINA achieves an improvement of 24.34% over LINC on the FOLIO dataset, while also surpassing prompting strategies like CoT and CoT-SC by up to 24.02%. Our code is available at https://github.com/wufeiwuwoshihua/nshy.

Figures (5)

• Figure 1: The framework of the LLM-driven Neuro-Symbolic Approach for Faithful Logical Reasoning consists of two main components: the Information Extraction Module and the LLM-driven Symbolic Reasoning Module. The close-ended reasoning questionon the left is processed by the Information Extraction Module to generate first-order logic statements ($LS$), natural language information ($NL$), and hypothesis ($H$). These outputs are then fed into the LLM-driven Symbolic Reasoning Module on the right, which performs deductive reasoning to derive the final answer.
• Figure 2: An illustration of the graph interpretation of LINA and prompt-based reasoning. Here $q$ denotes the question proposition, while $a$ denotes the option proposition. The $q'$ denotes $\neg q$, and the $a'$ denotes $\neg a$. The goal of LINA is to find a path from $q\land a$ to $q'$ or $a'$ in order to falsify$a$. The goal of prompt-based reasoning is to find a path from $q$ to $a$ in order to verify$a$.
• Figure 3: Comparison between LINA and SatLM on the ProofWriter and LogiQA dataset.
• Figure 4: Comparison between LINA and ToT on the ReClor and LogiQA dataset.
• Figure 5: A comparative case of LINC and LINA from the LogiQA dataset. Text highlighted in cyan represents different content expressed by the two methods, while text highlighted in yellow represents content that is unique to one of the methods.

Theorems & Definitions (3)

• Lemma 1
• Lemma 2
• Theorem 1