Towards an Application-Centric Benchmark Suite for Spatiotemporal Database Systems

TL;DR

The paper addresses the lack of end-to-end benchmarks for spatiotemporal database systems used in mobility-based applications. It argues for an application-centric benchmarking suite that models realistic client interactions, data scales, and query workloads to enable meaningful QoS comparisons. The authors define three key requirements—flexible workloads, platform extensibility, and independent analysis—and propose a modular three-phase design (Pre-, In-, Post-Experiment) with data generation, load orchestration, query translation, and analytics components. This framework aims to support both single-node and distributed deployments, facilitating realistic benchmarking, system selection, and performance understanding for moving object data workloads.

Abstract

Spatiotemporal data play a key role for mobility-based applications and are their produced volume is growing continuously, among others, due to the increased availability of IoT devices. When working with spatiotemporal data, developers rely on spatiotemporal database systems such as PostGIS or MobilityDB. For better understanding their quality of service behavior and then choosing the best system, benchmarking is the go-to approach. Unfortunately, existing work in this field studies only small isolated aspects and a comprehensive application-centric benchmark suite is still missing. In this paper, we argue that an application-centric benchmark suite for spatiotemporal database systems is urgently needed. We identify requirements for such a benchmark suite, discuss domain-specific challenges, and sketch-out the architecture of a modular benchmarking suite.

Figures (1)

• Figure 1: Each component is designed as a separate module with unique configuration options, which allows users to insert their own implementation. The analysis platform should be able to evaluate multiple experiments at once.