Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bridging Information Gaps in Dialogues With Grounded Exchanges Using Knowledge Graphs

Phillip Schneider, Nektarios Machner, Kristiina Jokinen, Florian Matthes

TL;DR

Bridging information gaps in dialogues with grounded exchanges using knowledge graphs tackles semantic misalignment between user language and system knowledge. The authors introduce BridgeKG, a JSON-LD grounded dataset across five domains and evaluate four LLMs on grounding-act classification and grounded-knowledge identification tasks. Results show that few-shot prompting improves grounding-act classification and that large models (e.g., GPT-4o) excel at identifying grounded knowledge within a knowledge-graph interface, while open-source models lag on structured data tasks. The work highlights knowledge graphs as semantic layers between unstructured dialogue and structured data and suggests a multi-component pipeline for end-to-end grounded dialogue systems. These findings provide practical guidance for building coherence-rich dialogue systems that align user knowledge with structured knowledge bases.

Abstract

Knowledge models are fundamental to dialogue systems for enabling conversational interactions, which require handling domain-specific knowledge. Ensuring effective communication in information-providing conversations entails aligning user understanding with the knowledge available to the system. However, dialogue systems often face challenges arising from semantic inconsistencies in how information is expressed in natural language compared to how it is represented within the system's internal knowledge. To address this problem, we study the potential of large language models for conversational grounding, a mechanism to bridge information gaps by establishing shared knowledge between dialogue participants. Our approach involves annotating human conversations across five knowledge domains to create a new dialogue corpus called BridgeKG. Through a series of experiments on this dataset, we empirically evaluate the capabilities of large language models in classifying grounding acts and identifying grounded information items within a knowledge graph structure. Our findings offer insights into how these models use in-context learning for conversational grounding tasks and common prediction errors, which we illustrate with examples from challenging dialogues. We discuss how the models handle knowledge graphs as a semantic layer between unstructured dialogue utterances and structured information items.

Bridging Information Gaps in Dialogues With Grounded Exchanges Using Knowledge Graphs

TL;DR

Bridging information gaps in dialogues with grounded exchanges using knowledge graphs tackles semantic misalignment between user language and system knowledge. The authors introduce BridgeKG, a JSON-LD grounded dataset across five domains and evaluate four LLMs on grounding-act classification and grounded-knowledge identification tasks. Results show that few-shot prompting improves grounding-act classification and that large models (e.g., GPT-4o) excel at identifying grounded knowledge within a knowledge-graph interface, while open-source models lag on structured data tasks. The work highlights knowledge graphs as semantic layers between unstructured dialogue and structured data and suggests a multi-component pipeline for end-to-end grounded dialogue systems. These findings provide practical guidance for building coherence-rich dialogue systems that align user knowledge with structured knowledge bases.

Abstract

Knowledge models are fundamental to dialogue systems for enabling conversational interactions, which require handling domain-specific knowledge. Ensuring effective communication in information-providing conversations entails aligning user understanding with the knowledge available to the system. However, dialogue systems often face challenges arising from semantic inconsistencies in how information is expressed in natural language compared to how it is represented within the system's internal knowledge. To address this problem, we study the potential of large language models for conversational grounding, a mechanism to bridge information gaps by establishing shared knowledge between dialogue participants. Our approach involves annotating human conversations across five knowledge domains to create a new dialogue corpus called BridgeKG. Through a series of experiments on this dataset, we empirically evaluate the capabilities of large language models in classifying grounding acts and identifying grounded information items within a knowledge graph structure. Our findings offer insights into how these models use in-context learning for conversational grounding tasks and common prediction errors, which we illustrate with examples from challenging dialogues. We discuss how the models handle knowledge graphs as a semantic layer between unstructured dialogue utterances and structured information items.
Paper Structure (12 sections, 3 figures, 7 tables)

This paper contains 12 sections, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Dataset Annotation
  5. Experimental Setup
  6. Results and Discussion
  7. Classification of Grounding Acts
  8. Identification of Grounded Knowledge
  9. Conclusion and Future Work
  10. Limitations
  11. Ethical Considerations
  12. Appendix

Figures (3)

  • Figure 1: Performance comparison of precision, recall, and F1-score by grounding act for the Llama-3-70B model with all input utterances (n=all).
  • Figure 2: Count of predictions in JSON-LD format with valid properties, valid values, or identical content for evaluated models in zero- (Z) and few-shot (F) settings.
  • Figure 3: Confusion matrices for few-shot classification results of GPT-4o with three input utterances and Llama-3-70B with all input utterances.