ConvoSense: Overcoming Monotonous Commonsense Inferences for Conversational AI

TL;DR

ConvoSense tackles the challenge of enriching conversational AI with deep, context-aware commonsense by creating a large-scale GPT-generated dataset with 10 inference types, totaling over 500k inferences across 12k dialogues. It introduces a rigorous evaluation framework showing GPT-generated inferences can match or surpass human baselines in reasonability while delivering substantially higher novelty and detail, aided by diverse-generation strategies such as diverse beam search. The authors provide a structured pipeline (dialogue/utterance selection, ten inference types, and quality checks) and compare against human-generated datasets, establishing ConvoSense as a scalable, open-source resource for training more capable dialogue systems. The work also analyzes limitations and offers future directions for improving diversity in distilled models and integrating commonsense reasoning into downstream dialogue tasks.

Abstract

Mastering commonsense understanding and reasoning is a pivotal skill essential for conducting engaging conversations. While there have been several attempts to create datasets that facilitate commonsense inferences in dialogue contexts, existing datasets tend to lack in-depth details, restate information already present in the conversation, and often fail to capture the multifaceted nature of commonsense reasoning. In response to these limitations, we compile a new synthetic dataset for commonsense reasoning in dialogue contexts using GPT, ConvoSense, that boasts greater contextual novelty, offers a higher volume of inferences per example, and substantially enriches the detail conveyed by the inferences. Our dataset contains over 500,000 inferences across 12,000 dialogues with 10 popular inference types, which empowers the training of generative commonsense models for dialogue that are superior in producing plausible inferences with high novelty when compared to models trained on the previous datasets. To the best of our knowledge, ConvoSense is the first of its kind to provide such a multitude of novel inferences at such a large scale.

Figures (3)

• Figure 1: Cause and Attribute inferences written by humans (top, green) and generated by GPT (bottom, blue).
• Figure 2: Average % of new & detailed inferences out of all positive novelty inferences for each data source.
• Figure 3: Desire and Desire$_o$ inferences in the $\mathbb{C}$onvo$\mathbb{S}$ense dataset.