Global Benchmark Database
Ashlin Iser, Christoph Jabs
TL;DR
Global Benchmark Database (GBD) addresses the need for sustainable, cross-domain access to benchmark instances and their metadata. It introduces a context-based data model with one-to-one and one-to-many features, plus context mappings and a flexible query language, complemented by CLI, web REST, and Python interfaces for provisioning, querying, feature extraction, and transformation. The work details workflows for data-source configuration, database initialization, feature extraction, and cross-context instance transformations, along with extension mechanisms to incorporate new domains. By enabling isomorphism-aware deduplication through isohash and rich cross-domain joins, GBD supports reproducible, scalable data-driven benchmarking and analysis across SAT, MaxSAT, and PBO, with potential to inform explainable AI approaches in algorithm selection and evaluation.
Abstract
This paper presents Global Benchmark Database (GBD), a comprehensive suite of tools for provisioning and sustainably maintaining benchmark instances and their metadata. The availability of benchmark metadata is essential for many tasks in empirical research, e.g., for the data-driven compilation of benchmarks, the domain-specific analysis of runtime experiments, or the instance-specific selection of solvers. In this paper, we introduce the data model of GBD as well as its interfaces and provide examples of how to interact with them. We also demonstrate the integration of custom data sources and explain how to extend GBD with additional problem domains, instance formats and feature extractors.