Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Chain of Condition: Construct, Verify and Solve Conditions for Conditional Question Answering

Jiuheng Lin, Yuxuan Lai, Yansong Feng

TL;DR

A novel prompting approach, Chain of condition, by first identifying all conditions and constructing their logical relationships explicitly according to the document, then verifying whether these conditions are satisfied, finally solving the logical expression to indicate any missing conditions and generating the answer accordingly.

Abstract

Conditional question answering (CQA) is an important task that aims to find probable answers and identify missing conditions. Existing approaches struggle with CQA due to two challenges: (1) precisely identifying necessary conditions and the logical relationship, and (2) verifying conditions to detect any that are missing. In this paper, we propose a novel prompting approach, Chain of condition, by first identifying all conditions and constructing their logical relationships explicitly according to the document, then verifying whether these conditions are satisfied, finally solving the logical expression to indicate any missing conditions and generating the answer accordingly. Experiments on two CQA benchmark datasets show our chain of condition outperforms existing prompting baselines, establishing a new state of the art. Furthermore, with only a few examples, our method can facilitate GPT-3.5-Turbo or GPT-4 to outperform all existing supervised models.

Chain of Condition: Construct, Verify and Solve Conditions for Conditional Question Answering

TL;DR

A novel prompting approach, Chain of condition, by first identifying all conditions and constructing their logical relationships explicitly according to the document, then verifying whether these conditions are satisfied, finally solving the logical expression to indicate any missing conditions and generating the answer accordingly.

Abstract

Conditional question answering (CQA) is an important task that aims to find probable answers and identify missing conditions. Existing approaches struggle with CQA due to two challenges: (1) precisely identifying necessary conditions and the logical relationship, and (2) verifying conditions to detect any that are missing. In this paper, we propose a novel prompting approach, Chain of condition, by first identifying all conditions and constructing their logical relationships explicitly according to the document, then verifying whether these conditions are satisfied, finally solving the logical expression to indicate any missing conditions and generating the answer accordingly. Experiments on two CQA benchmark datasets show our chain of condition outperforms existing prompting baselines, establishing a new state of the art. Furthermore, with only a few examples, our method can facilitate GPT-3.5-Turbo or GPT-4 to outperform all existing supervised models.
Paper Structure (34 sections, 2 equations, 2 figures, 23 tables)

This paper contains 34 sections, 2 equations, 2 figures, 23 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminary
  4. Methodology
  5. Condition Identification and Logical Relationship Construction
  6. Condition Verification
  7. Expression Solution and Answer Generation
  8. Experimental Setup
  9. Datasets and Evaluation Metrics
  10. Baselines
  11. Prompting Baselines
  12. Supervised Baselines
  13. LLM Setup
  14. Results and Analysis
  15. Main Results
  16. ...and 19 more sections

Figures (2)

  • Figure 1: An example of conditional question answering. All conditions are colored. The conditions in green are satisfied by the user's description, while those in red are not mentioned. The second red condition, has lived together for at least 3 years, is not necessary because it has an "either" relationship with an already satisfied condition. But the other two red conditions are required to make the answer "up to $120000" correct.
  • Figure 2: Method overview. Chain of condition consists of three main steps: condition identification and logical construction, condition verification, expression solution and answer generation.