Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conversational Lexicography: Querying Lexicographic Data on Knowledge Graphs with SPARQL through Natural Language

Kilian Sennrich, Sina Ahmadi

TL;DR

This work advances natural language access to lexicographic data on knowledge graphs by introducing a four-dimension taxonomy for Wikidata's lexicographic ontology and a template-driven dataset with over 1.2 million NL-to-SPARQL mappings. It evaluates modest-size transformer models (GPT-2, Phi-1.5) and a strong baseline (GPT-3.5-Turbo), revealing that model size and diverse pre-training critically influence generalization to unseen query structures. The study demonstrates that while GPT-3.5-Turbo generalizes better than smaller models, robust generalization and scalability for complex lexicographic queries remain challenging. The findings highlight practical implications for deploying NL interfaces to KGs and point to future work in larger, reasoning-focused models and cross-KG applicability.

Abstract

Knowledge graphs offer an excellent solution for representing the lexical-semantic structures of lexicographic data. However, working with the SPARQL query language represents a considerable hurdle for many non-expert users who could benefit from the advantages of this technology. This paper addresses the challenge of creating natural language interfaces for lexicographic data retrieval on knowledge graphs such as Wikidata. We develop a multidimensional taxonomy capturing the complexity of Wikidata's lexicographic data ontology module through four dimensions and create a template-based dataset with over 1.2 million mappings from natural language utterances to SPARQL queries. Our experiments with GPT-2 (124M), Phi-1.5 (1.3B), and GPT-3.5-Turbo reveal significant differences in model capabilities. While all models perform well on familiar patterns, only GPT-3.5-Turbo demonstrates meaningful generalization capabilities, suggesting that model size and diverse pre-training are crucial for adaptability in this domain. However, significant challenges remain in achieving robust generalization, handling diverse linguistic data, and developing scalable solutions that can accommodate the full complexity of lexicographic knowledge representation.

Conversational Lexicography: Querying Lexicographic Data on Knowledge Graphs with SPARQL through Natural Language

TL;DR

This work advances natural language access to lexicographic data on knowledge graphs by introducing a four-dimension taxonomy for Wikidata's lexicographic ontology and a template-driven dataset with over 1.2 million NL-to-SPARQL mappings. It evaluates modest-size transformer models (GPT-2, Phi-1.5) and a strong baseline (GPT-3.5-Turbo), revealing that model size and diverse pre-training critically influence generalization to unseen query structures. The study demonstrates that while GPT-3.5-Turbo generalizes better than smaller models, robust generalization and scalability for complex lexicographic queries remain challenging. The findings highlight practical implications for deploying NL interfaces to KGs and point to future work in larger, reasoning-focused models and cross-KG applicability.

Abstract

Knowledge graphs offer an excellent solution for representing the lexical-semantic structures of lexicographic data. However, working with the SPARQL query language represents a considerable hurdle for many non-expert users who could benefit from the advantages of this technology. This paper addresses the challenge of creating natural language interfaces for lexicographic data retrieval on knowledge graphs such as Wikidata. We develop a multidimensional taxonomy capturing the complexity of Wikidata's lexicographic data ontology module through four dimensions and create a template-based dataset with over 1.2 million mappings from natural language utterances to SPARQL queries. Our experiments with GPT-2 (124M), Phi-1.5 (1.3B), and GPT-3.5-Turbo reveal significant differences in model capabilities. While all models perform well on familiar patterns, only GPT-3.5-Turbo demonstrates meaningful generalization capabilities, suggesting that model size and diverse pre-training are crucial for adaptability in this domain. However, significant challenges remain in achieving robust generalization, handling diverse linguistic data, and developing scalable solutions that can accommodate the full complexity of lexicographic knowledge representation.

Paper Structure

This paper contains 38 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Datasets
  4. Generation
  5. Evaluation
  6. Methodology
  7. Taxonomy for the Lexicographic Data
  8. D1: Lexical Properties
  9. D2: Single vs. Multi Lexeme Output
  10. D3: Mono- vs. Multilinguality
  11. D4: Simple vs. Complex Queries
  12. Implementation
  13. Evaluation
  14. Dataset
  15. Template Sources
  16. ...and 23 more sections

Figures (3)

  • Figure 1: Conversational lexicography: enabling natural language queries to KGs by automatically generating SPARQL code, eliminating the need for manual query writing
  • Figure 2: Our approach to creating SPARQL templates based on a four-dimension taxonomy followed by dataset population and model implementation. The ultimate goal is to infer the models by querying in natural language.
  • Figure 3: Distribution of the number of populated data tuples per template