LST-Bench: Benchmarking Log-Structured Tables in the Cloud
Jesús Camacho-Rodríguez, Ashvin Agrawal, Anja Gruenheid, Ashit Gosalia, Cristian Petculescu, Josep Aguilar-Saborit, Avrilia Floratou, Carlo Curino, Raghu Ramakrishnan
TL;DR
Log-Structured Tables (LSTs) provide mutability for analytics atop immutable columnar formats, addressing update-heavy workloads that Parquet/ORC struggle with. The authors propose LST-Bench, a modular benchmarking framework that extends TPC-DS with LST-specific workload patterns and a novel stability metric, and demonstrate its use across Delta Lake, Iceberg, and Hudi on Spark and Trino. They introduce a three-dimensional conceptual model (metadata, algorithms, engines) to connect LST design choices with query performance, and reveal trade-offs between Copy-on-Write and Merge-on-Read, as well as engine-dependent behavior in terms of maintenance, concurrency, and time-travel. The work delivers an open-source, extensible toolchain for fair, repeatable LST evaluation in cloud environments and highlights practical guidance for tuning LST deployments while identifying avenues for broader workload coverage and automated deployment in future research.
Abstract
Data processing engines increasingly leverage distributed file systems for scalable, cost-effective storage. While the Apache Parquet columnar format has become a popular choice for data storage and retrieval, the immutability of Parquet files renders it impractical to meet the demands of frequent updates in contemporary analytical workloads. Log-Structured Tables (LSTs), such as Delta Lake, Apache Iceberg, and Apache Hudi, offer an alternative for scenarios requiring data mutability, providing a balance between efficient updates and the benefits of columnar storage. They provide features like transactions, time-travel, and schema evolution, enhancing usability and enabling access from multiple engines. Moreover, engines like Apache Spark and Trino can be configured to leverage the optimizations and controls offered by LSTs to meet specific business needs. Conventional benchmarks and tools are inadequate for evaluating the transformative changes in the storage layer resulting from these advancements, as they do not allow us to measure the impact of design and optimization choices in this new setting. In this paper, we propose a novel benchmarking approach and metrics that build upon existing benchmarks, aiming to systematically assess LSTs. We develop a framework, LST-Bench, which facilitates effective exploration and evaluation of the collaborative functioning of LSTs and data processing engines through tailored benchmark packages. A package is a mix of use patterns reflecting a target workload; LST-Bench makes it easy to define a wide range of use patterns and combine them into a package, and we include a baseline package for completeness. Our assessment demonstrates the effectiveness of our framework and benchmark packages in extracting valuable insights across diverse environments. The code for LST-Bench is open-sourced and is available at https://github.com/microsoft/lst-bench/ .