Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Complex Reasoning over Logical Queries on Commonsense Knowledge Graphs

Tianqing Fang, Zeming Chen, Yangqiu Song, Antoine Bosselut

TL;DR

The paper introduces COM2, a large-scale dataset for complex commonsense reasoning by sampling multi-hop conjunctive logical queries from commonsense knowledge graphs and verbalizing them into narrative questions. It densifies the graph, filters low-quality triples with a plausibility scorer, and uses both rule-based and LLM-driven verbalization to create fluent narrative contexts, yielding 790K training examples plus a 1.3K manually verified evaluation set. Fine-tuning QA and generation models on COM2 yields substantial improvements across eight zero-shot commonsense benchmarks, with notable gains in both in-domain and out-of-domain tasks and strong zero-shot transfer relative to larger models. The work demonstrates scalable, minimally supervised acquisition of complex reasoning signals from CSKGs and provides code and data for broad use.

Abstract

Event commonsense reasoning requires the ability to reason about the relationship between events, as well as infer implicit context underlying that relationship. However, data scarcity makes it challenging for language models to learn to generate commonsense inferences for contexts and questions involving interactions between complex events. To address this demand, we present COM2 (COMplex COMmonsense), a new dataset created by sampling multi-hop logical queries (e.g., the joint effect or cause of both event A and B, or the effect of the effect of event C) from an existing commonsense knowledge graph (CSKG), and verbalizing them using handcrafted rules and large language models into multiple-choice and text generation questions. Our experiments show that language models trained on COM2 exhibit significant improvements in complex reasoning ability, resulting in enhanced zero-shot performance in both in-domain and out-of-domain tasks for question answering and generative commonsense reasoning, without expensive human annotations. Code and data are available at https://github.com/tqfang/complex-commonsense-reasoning.

Complex Reasoning over Logical Queries on Commonsense Knowledge Graphs

TL;DR

The paper introduces COM2, a large-scale dataset for complex commonsense reasoning by sampling multi-hop conjunctive logical queries from commonsense knowledge graphs and verbalizing them into narrative questions. It densifies the graph, filters low-quality triples with a plausibility scorer, and uses both rule-based and LLM-driven verbalization to create fluent narrative contexts, yielding 790K training examples plus a 1.3K manually verified evaluation set. Fine-tuning QA and generation models on COM2 yields substantial improvements across eight zero-shot commonsense benchmarks, with notable gains in both in-domain and out-of-domain tasks and strong zero-shot transfer relative to larger models. The work demonstrates scalable, minimally supervised acquisition of complex reasoning signals from CSKGs and provides code and data for broad use.

Abstract

Event commonsense reasoning requires the ability to reason about the relationship between events, as well as infer implicit context underlying that relationship. However, data scarcity makes it challenging for language models to learn to generate commonsense inferences for contexts and questions involving interactions between complex events. To address this demand, we present COM2 (COMplex COMmonsense), a new dataset created by sampling multi-hop logical queries (e.g., the joint effect or cause of both event A and B, or the effect of the effect of event C) from an existing commonsense knowledge graph (CSKG), and verbalizing them using handcrafted rules and large language models into multiple-choice and text generation questions. Our experiments show that language models trained on COM2 exhibit significant improvements in complex reasoning ability, resulting in enhanced zero-shot performance in both in-domain and out-of-domain tasks for question answering and generative commonsense reasoning, without expensive human annotations. Code and data are available at https://github.com/tqfang/complex-commonsense-reasoning.
Paper Structure (56 sections, 3 equations, 5 figures, 12 tables)

This paper contains 56 sections, 3 equations, 5 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Complex Logical Queries
  4. Complex Commonsense Reasoning
  5. Methodology
  6. Pre-processing
  7. Sparsity
  8. Quality
  9. Query Sampling
  10. Distractor Sampling
  11. Verbalization
  12. Anchor Entity Verbalization
  13. Relation Verbalization
  14. Human Annotation
  15. Quality
  16. ...and 41 more sections

Figures (5)

  • Figure 1: An example of conjunctive logical queries and their verbalization as complex commonsense inferences.
  • Figure 2: Overview of the construction process. $f$ represents a verbalization function for the context, and $g$ represents the one for the question.
  • Figure 3: Visualization of query structures. The anchor entities and relations are specified to instantiate the query. 'p' and 'i' represent projection and intersection, and the number ahead of p and i indicates the number of anchor entities and free variables.
  • Figure 4: Examples of different query types, their verbalization, and corresponding questions.
  • Figure 5: Annotation interface.