Baseline: Operation-Based Evolution and Versioning of Data
Jonathan Edwards, Tomas Petricek
TL;DR
Baseline proposes an operation-based framework for data evolution across time, space, and design changes by introducing Operational Differencing, which uses Project and Retract to diffs across divergent timelines. The approach unifies data manipulation with schema migration, enabling fine-grained diffing, merging, and cross-branch transfers without a central repository. It extends from rich data models to databases and even to updatable views via operationalized queries, and demonstrates partial success on four of eight Schema Evolution challenges. The work highlights future directions including a declarative rule language, a set-like datatype to address deduplication issues, and a fully fleshed-out operationalized query language. Overall, Baseline offers a novel, data-centric counterpart to traditional version control that accommodates schema changes and collaborative manipulation with explicit operation histories.
Abstract
Baseline is a platform for richly structured data supporting change in multiple dimensions: mutation over time, collaboration across space, and evolution through design changes. It is built upon Operational Differencing, a new technique for managing data in terms of high-level operations that include refactorings and schema changes. We use operational differencing to construct an operation-based form of version control on data structures used in programming languages and relational databases. This approach to data version control does fine-grained diffing and merging despite intervening structural transformations like schema changes. It offers users a simplified conceptual model of version control for ad hoc usage: There is no repo; Branching is just copying. The informaton maintained in a repo can be synthesized more precisely from the append-only histories of branches. Branches can be flexibly shared as is commonly done with document files, except with the added benefit of diffing and merging. We conjecture that queries can be operationalized into a sequence of schema and data operations. We develop that idea on a query language fragment containing selects and joins. Operationalized queries are represented as a future timeline that is speculatively executed as a branch off of the present state, returning a value from its hypothetical future. Operationalized queries get rewritten to accommodate schema change "for free" by the machinery of operational differencing. Altogether we develop solutions to four of the eight challenge problems of schema evolution identified in a recent paper.