SchemaDB: Structures in Relational Datasets
Cody James Christopher, Kristen Moore, David Liebowitz
TL;DR
The paper tackles the scarcity of publicly available real-world relational schemas by introducing SchemaDB, a dataset of 2,500 standardized schemata derived from public GitHub repositories and converted to a canonical MySQL form. It details a robust ETL pipeline, including collection, filtration, dialect translation, graph-based representation, skeleton extraction, and duplication handling via MOSS, along with heuristic augmentation to recover missing foreign keys. The work provides extensive analytics on schema structure and demonstrates potential research avenues in normalisation analysis, entity semantics, and generative schema construction for ML and cyber deception. By releasing the dataset and tooling, the authors aim to catalyse ML-driven schema generation, benchmarking, and realistic synthetic database content for education and cyber-range applications, while noting representativeness limitations of purely public sources.
Abstract
In this paper we introduce the SchemaDB data-set; a collection of relational database schemata in both sql and graph formats. Databases are not commonly shared publicly for reasons of privacy and security, so schemata are not available for study. Consequently, an understanding of database structures in the wild is lacking, and most examples found publicly belong to common development frameworks or are derived from textbooks or engine benchmark designs. SchemaDB contains 2,500 samples of relational schemata found in public repositories which we have standardised to MySQL syntax. We provide our gathering and transformation methodology, summary statistics, and structural analysis, and discuss potential downstream research tasks in several domains.