IFDNS: An Iterative Feedback-Driven Neuro-Symbolic Method for Faithful Logical Reasoning

TL;DR

IFDNS addresses faithfulness and information loss in LLM-driven logical reasoning by introducing a multi-round feedback loop that converts natural language contexts into propositional symbols and implication expressions, extends them with a Python deductive engine, and reintegrates the results into prompts through translation and word-order optimization. The framework is orthogonal to standard prompting approaches and improves CoT and CoT-SC performance across seven datasets, including gains such as +9.40% on LogiQA and +11.70% on PrOntoQA. Across six logical datasets plus a mathematical-premise-order dataset, IFDNS consistently boosts accuracy over baselines and demonstrates stronger gains on more challenging tasks. These results suggest a practical pathway to more faithful, multi-hop logical reasoning in LLMs using iterative neuro-symbolic prompting.

Abstract

Large language models (LLMs) have demonstrated impressive capabilities across a wide range of reasoning tasks, including logical and mathematical problem-solving. While prompt-based methods like Chain-of-Thought (CoT) can enhance LLM reasoning abilities to some extent, they often suffer from a lack of faithfulness, where the derived conclusions may not align with the generated reasoning chain. To address this issue, researchers have explored neuro-symbolic approaches to bolster LLM logical reasoning capabilities. However, existing neuro-symbolic methods still face challenges with information loss during the process. To overcome these limitations, we introduce Iterative Feedback-Driven Neuro-Symbolic (IFDNS), a novel prompt-based method that employs a multi-round feedback mechanism to address LLM limitations in handling complex logical relationships. IFDNS utilizes iterative feedback during the logic extraction phase to accurately extract causal relationship statements and translate them into propositional and logical implication expressions, effectively mitigating information loss issues. Furthermore, IFDNS is orthogonal to existing prompt methods, allowing for seamless integration with various prompting approaches. Empirical evaluations across six datasets demonstrate the effectiveness of IFDNS in significantly improving the performance of CoT and Chain-of-Thought with Self-Consistency (CoT-SC). Specifically, IFDNS achieves a +9.40% accuracy boost for CoT on the LogiQA dataset and a +11.70% improvement for CoT-SC on the PrOntoQA dataset.

Figures (4)

• Figure 1: In the LoT case from the ProofWriter dataset, LoT firstly fails to comprehensively and accurately extract causal relationship statements. Secondly, it lacks the capability to fully interpret all propositions within the context and translate them into corresponding implication expressions, thereby preventing logical extension.
• Figure 2: The IFDNS framework operates through five sequential stages: (1) Upon receiving MCQA input, LLMs extract causal relationship statements from the context; (2) Extract propositions and implication expressions from causal relationship statements, while conducting multiple rounds of feedback for both the first and second stages. The statements highlighted in red in the diagram are those that are about to be modified or removed after the feedback process; (3) Python-based deductive engine extends implications via logical inference laws; (4) Translation of extended implications into natural language for context augmentation; (5) Word order reordering module optimizes context sequencing to enable LLM-driven end-to-end reasoning.
• Figure 3: Comparison between CoT+IFDNS and Logic-LM
• Figure 4: Comparison case study between IFDNS and LoT from the ProofWriter dataset. The red dashed rectangles highlight the differences between the two methods. LoT lacks the ability to understand all the propositions contained in the context and extract corresponding implication expressions, thus hindering logical extension. In contrast, IFDNS performs multi-round feedback on each stage of causal statement extraction, proposition and implication expression extraction. It utilizes broader logical deduction laws to extract implicit relationships and seamlessly integrates the additional information back into the original prompt, ultimately enhancing the LLM’s ability to generate accurate results.