Making Implicit Premises Explicit in Logical Understanding of Enthymemes

TL;DR

A pipeline that integrates a large language model to generate intermediate implicit premises based on the explicit premise and claim, a neuro-symbolic reasoner based on a SAT solver to determine entailment, and a neuro-symbolic reasoner based on a SAT solver to determine entailment is proposed.

Abstract

Real-world arguments in text and dialogues are normally enthymemes (i.e. some of their premises and/or claims are implicit). Natural language processing (NLP) methods for handling enthymemes can potentially identify enthymemes in text but they do not decode their underlying logic, whereas logic-based approaches for handling them assume a knowledgebase with sufficient formulae that can be used to decode them via abduction. There is therefore a lack of a systematic method for translating textual components of an enthymeme into a logical argument and generating the logical formulae required for their decoding, and thereby showing logical entailment. To address this, we propose a pipeline that integrates: (1) a large language model (LLM) to generate intermediate implicit premises based on the explicit premise and claim; (2) another LLM to translate the natural language into logical formulas; and (3) a neuro-symbolic reasoner based on a SAT solver to determine entailment. We evaluate our pipeline on two enthymeme datasets, demonstrating promising performance in selecting the correct implicit premise, as measured by precision, recall, F1-score, and accuracy.

Figures (11)

• Figure 1: AMR for the sentence "The boy wants to go." (left) and "The boy does not want to go." (right).
• Figure 2: Our neuro-symbolic pipeline where input is a set of natural language sentences (e.g. premise, implicit premises and claim), and the output is a label.
• Figure 3: Overall accuracy (entailment and non-entailment) for different options for the implicit premises for the ANLI and ARCT datasets. Original means that the helpful/unhelpful intermediate premises given in the dataset, One- (resp. Two- and Three-) step means using the response from prompting the LLM for one (resp. two and three) steps of helpful/unhelpful intermediate premises.
• Figure 4: A structured argument graph analyzing possible causes for a torn spiderweb. The claim suggests a large insect escaped recently, which is supported by Implicit Premise A (relating torn webs to prey escape) but contradicted by Implicit Premise B (small insect fled). Implicit Premise C (environmental damage) and the premise alone (torn web) remain neutral to the claim. The labels on the arcs show neuro-matching relations that hold for the support relation, and the neuro-contradiction relation that holds for the contradiction relation. In this example, Implicit Premise A together with the neuro-matching relation provides the decoding of the enthymeme. So adding the AMR formulae for the Implicit Premise A to the AMR formulae for the Premise, plus the neuro-matching relations, entails the claim. The black arrow from the Premise to a implicit premise denotes the combination of the Premise and implicit premise.
• Figure 5: F1-score by Different Steps of Implicit Premise of the ANLI dataset

Theorems & Definitions (16)

• Definition 1
• Example 1
• Definition 2
• Definition 3
• Example 2
• Definition 4
• Example 3
• Definition 5
• Definition 6
• Example 4