CauseJudger: Identifying the Cause with LLMs for Abductive Logical Reasoning

TL;DR

This work proposes a new framework for LLMs abductive logical reasoning called CauseJudger (CJ), which identifies the authenticity of possible cause by transforming thinking from reverse to forward and removing irrelevant information.

Abstract

Large language models (LLMs) have been utilized in solving diverse reasoning tasks, encompassing common sense, arithmetic and deduction tasks. However, with difficulties of reversing thinking patterns and irrelevant premises, how to determine the authenticity of the cause in abductive logical reasoning remains underexplored. Inspired by hypothesis and verification method and identification of irrelevant information in human thinking process, we propose a new framework for LLMs abductive logical reasoning called CauseJudger (CJ), which identifies the authenticity of possible cause by transforming thinking from reverse to forward and removing irrelevant information. In addition, we construct an abductive logical reasoning dataset for decision task called CauseLogics, which contains 200,000 tasks of varying reasoning lengths. Our experiments show the efficiency of CJ with overall experiments and ablation experiments as well as case studies on our dataset and reconstructed public dataset. Notably, CJ's implementation is efficient, requiring only two calls to LLM. Its impact is profound: when using gpt-3.5, CJ achieves a maximum correctness improvement of 41% compared to Zero-Shot-CoT. Moreover, with gpt-4, CJ attains an accuracy exceeding 90% across all datasets.

Figures (5)

• Figure 1: Overview of our study. We analyse the characteristics of abductive logical reasoning and construct CauseLogics dataset. We propose CauseJudger framework for LLMs abductive logical reasoning, with three stages: (a) Change the reverse thinking pattern into a forward one; (b) Remove the irrelevant information ; (c) Do forward reasoning to make final judgment.
• Figure 2: Difference in reasoning pattern between deductive logical reasoning and abductive logical reasoning.
• Figure 3: Overview of CauseJudger. We use circles of different colors to represent different elements, with blue circles representing premises, yellow circles representing rules, green circles representing target phenomena, purple circles representing possible cause, and gray circle representing intermediate results of reasoning. CauseJudger is divided into three stages, using hypothesis and verification method to reverse the thinking pattern, remove irrelevant information, and then reasoning forwardly to solve the abductive logical reasoning problem.
• Figure 4: Steps of the potential usage of CauseJudger on possibility report of diseases.
• Figure 5: Detailed example of the potential usage of CauseJudger on possibility report of diseases.