Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large Language Models for JSON Schema Discovery

Michael J. Mior

TL;DR

The paper tackles the problem that JSON schema discovery often yields structurally correct but semantically weak schemas. It introduces a three-pronged approach powered by large language models: generating natural language descriptions for schema elements, producing semantically meaningful names for repeated definitions, and a classifier to select which properties are genuinely useful. Trained on 657 JSON Schemas from the JSON Schema Store and fine-tuned with Code Llama (via LoRA), the method demonstrates superior performance on standard text-generation metrics (e.g., BERTScore, ROUGE-L, BLEU), semantic naming (VarCLR), and property selection accuracy (~90.5%). The results suggest that semantic augmentation can meaningfully improve the usability and interoperability of automatically discovered schemas, and the approach can be integrated with existing schema discovery tools. Future work may broaden context usage and allow users to adjust the level of detail in the produced schemas.

Abstract

Semi-structured data formats such as JSON have proved to be useful data models for applications that require flexibility in the format of data stored. However, JSON data often come without the schemas that are typically available with relational data. This has resulted in a number of tools for discovering schemas from a collection of data. Although such tools can be useful, existing approaches focus on the syntax of documents and ignore semantic information. In this work, we explore the automatic addition of meaningful semantic information to discovered schemas similar to information that is added by human schema authors. We leverage large language models and a corpus of manually authored JSON Schema documents to generate natural language descriptions of schema elements, meaningful names for reusable definitions, and identify which discovered properties are most useful and which can be considered "noise". Our approach performs well on existing metrics for text generation that have been previously shown to correlate well with human judgement.

Large Language Models for JSON Schema Discovery

TL;DR

The paper tackles the problem that JSON schema discovery often yields structurally correct but semantically weak schemas. It introduces a three-pronged approach powered by large language models: generating natural language descriptions for schema elements, producing semantically meaningful names for repeated definitions, and a classifier to select which properties are genuinely useful. Trained on 657 JSON Schemas from the JSON Schema Store and fine-tuned with Code Llama (via LoRA), the method demonstrates superior performance on standard text-generation metrics (e.g., BERTScore, ROUGE-L, BLEU), semantic naming (VarCLR), and property selection accuracy (~90.5%). The results suggest that semantic augmentation can meaningfully improve the usability and interoperability of automatically discovered schemas, and the approach can be integrated with existing schema discovery tools. Future work may broaden context usage and allow users to adjust the level of detail in the produced schemas.

Abstract

Semi-structured data formats such as JSON have proved to be useful data models for applications that require flexibility in the format of data stored. However, JSON data often come without the schemas that are typically available with relational data. This has resulted in a number of tools for discovering schemas from a collection of data. Although such tools can be useful, existing approaches focus on the syntax of documents and ignore semantic information. In this work, we explore the automatic addition of meaningful semantic information to discovered schemas similar to information that is added by human schema authors. We leverage large language models and a corpus of manually authored JSON Schema documents to generate natural language descriptions of schema elements, meaningful names for reusable definitions, and identify which discovered properties are most useful and which can be considered "noise". Our approach performs well on existing metrics for text generation that have been previously shown to correlate well with human judgement.
Paper Structure (15 sections, 6 figures, 3 tables)

This paper contains 15 sections, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Schema Discovery
  4. Large Language Models
  5. Schema Annotation
  6. Natural Language Descriptions
  7. Structure Naming
  8. Property Selection
  9. Methodology
  10. Evaluation
  11. Natural Language Descriptions
  12. Definitions
  13. Property Selection
  14. Related Work
  15. Conclusions and Future Work

Figures (6)

  • Figure 1: Original schema
  • Figure 2: Discovered schema
  • Figure 4: Example schema fragments for training
  • Figure 5: Examples used for property selection
  • Figure 6: Example descriptions generated by our model
  • ...and 1 more figures