Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GeoBenchr: An Application-Centric Benchmarking Suite for Spatiotemporal Database Platforms

Tim C. Rese, Nils Japke, Diana Baumann, Natalie Carl, David Bermbach

TL;DR

The proposed GeoBenchr enables comprehensive evaluation across diverse datasets, query types, and workload patterns, reflecting realistic use cases from domains such as cycling, aviation, and maritime tracking and highlights the importance of application-centric benchmarking in selecting suitable spatiotemporal database systems for real-world scenarios.

Abstract

The rapid growth of spatiotemporal data volumes needs to be handled by database systems capable of efficiently managing and querying such data. Existing systems such as PostGIS, SpaceTime, and MobilityDB offer partial solutions but differ widely in scope and performance. Also, first spatiotemporal benchmarks provide valuable insights but are limited in scope and, to our knowledge, no application-centric benchmarking suite exists. In this paper, we propose GeoBenchr, an open-source, application-centric benchmarking suite for spatiotemporal platforms. GeoBenchr enables comprehensive evaluation across diverse datasets, query types, and workload patterns, reflecting realistic use cases from domains such as cycling, aviation, and maritime tracking. We use our GeoBenchr prototype to evaluate several system aspects including scalability, configuration impact, and cross-platform performance comparison. Our results highlight the importance of application-centric benchmarking in selecting suitable spatiotemporal database systems for real-world scenarios.

GeoBenchr: An Application-Centric Benchmarking Suite for Spatiotemporal Database Platforms

TL;DR

The proposed GeoBenchr enables comprehensive evaluation across diverse datasets, query types, and workload patterns, reflecting realistic use cases from domains such as cycling, aviation, and maritime tracking and highlights the importance of application-centric benchmarking in selecting suitable spatiotemporal database systems for real-world scenarios.

Abstract

The rapid growth of spatiotemporal data volumes needs to be handled by database systems capable of efficiently managing and querying such data. Existing systems such as PostGIS, SpaceTime, and MobilityDB offer partial solutions but differ widely in scope and performance. Also, first spatiotemporal benchmarks provide valuable insights but are limited in scope and, to our knowledge, no application-centric benchmarking suite exists. In this paper, we propose GeoBenchr, an open-source, application-centric benchmarking suite for spatiotemporal platforms. GeoBenchr enables comprehensive evaluation across diverse datasets, query types, and workload patterns, reflecting realistic use cases from domains such as cycling, aviation, and maritime tracking. We use our GeoBenchr prototype to evaluate several system aspects including scalability, configuration impact, and cross-platform performance comparison. Our results highlight the importance of application-centric benchmarking in selecting suitable spatiotemporal database systems for real-world scenarios.
Paper Structure (27 sections, 5 figures, 2 tables)

This paper contains 27 sections, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background & Related Work
  3. Spatiotemporal and Moving Object Data
  4. Spatiotemporal Database Systems and Processing Engines
  5. Spatiotemporal Datasets And Generators
  6. Benchmarking Suites
  7. System Design
  8. GeoBenchr Architecture
  9. Application-Centric Benchmark Scenarios
  10. Flight
  11. Cycling
  12. AIS
  13. Evaluation
  14. Proof of Concept Implementation
  15. Supported SUTs
  16. ...and 12 more sections

Figures (5)

  • Figure 1: GeoBenchr's architecture is modular and allows the user to configure various parameters of the benchmark run, including dataset, scale factor, SUT, configuration profile, and workload parameters. Adapted and extended from rese2025towards.
  • Figure 2: The three real-world MOD datasets we base our application scenarios on, varying heavily in their characteristics such as data distribution and movement patterns. From left to right: Cycling data from the SimRa project karakaya2020simra, AIS data from the published Piraeus AIS dataset tritsarolis2022piraeus, and flight data from the Deutsche Flugsicherung (DFS).
  • Figure 3: We find the index choice between SP-GIST and GIST to have a negligible impact (on average, SPGIST is 0.67% slower) on our results. Time Partitioning can provide a benefit at times (1.13% faster on average), while spatial partitioning worsens overall performance in our evaluated cases, leading to a 38.11% performance degradation compared to the baseline of no partitioning.
  • Figure 4: Depending on the query, different SUTs excel. SedonaDB, while having the best overall performance, is outperformed by SpaceTime for some query/data scale combinations (further shown in \ref{['fig:platform-comparison']}). MobilityDB shows the advantage it has over PostGIS and TimeScaleDB in some of our queries, while being outperformed in others. Mind the log scale in Avi.Q3 and Avi.Q5. SedonaDB however has a significantly higher CPU and RAM footprint, requiring nearly 77% of available RAM on average during the experiment, compared to less than 8% for all other systems.
  • Figure 5: Empirical Cumulative Distribution Function of Query Durations across datasets. In some cases, database systems manage to outperform SedonaDB despite its in-memory architecture. SpaceTime shows strong performance across most queries, while TimeScaleDB and PostGIS also provide competitive performance in several queries. MobilityDB has the highest latency in our experiments, which is likely due to the large trip sizes for our ship datasets.