Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating NoSQL Databases for OLAP Workloads: A Benchmarking Study of MongoDB, Redis, Kudu and ArangoDB

Rishi Kesav Mohan, Risheek Rakshit Sukumar Kanmani, Krishna Anandan Ganesan, Nisha Ramasubramanian

TL;DR

This study investigates how NoSQL databases perform under OLAP workloads within a standardized data processing pipeline that includes HDFS, Spark, KoalaBench-derived inputs, and a BI layer. It benchmarks four NoSQL stores—MongoDB, Redis, ArangoDB, and Apache Kudu—plus PostgreSQL as a relational baseline, using Snowflake and Flat data models to reflect different schema designs. Across loading and five TPCH-like queries, the results show columnar storage (Apache Kudu) delivering the strongest performance, while other NoSQL options exhibit varied scalability and sometimes exponential growth in query times, especially with joins or complex aggregations. The findings support prioritizing columnar storage for scalable OLAP in NoSQL pipelines and point to future work integrating BI tools and expanding comparisons to additional columnar options.

Abstract

In the era of big data, conventional RDBMS models have become impractical for handling colossal workloads. Consequently, NoSQL databases have emerged as the preferred storage solutions for executing processing-intensive Online Analytical Processing (OLAP) tasks. Within the realm of NoSQL databases, various classifications exist based on their data storage mechanisms, making it challenging to select the most suitable one for a given OLAP workload. While each NoSQL database boasts distinct advantages, inherent scalability, adaptability to diverse data formats, and high data availability are universally recognized benefits crucial for managing OLAP workloads effectively. Existing research predominantly evaluates individual databases within custom data pipeline setups, lacking a standardized approach for comparative analysis across different databases to identify the optimal data pipeline for OLAP workloads. In this paper, we present our experimental insights into how various NoSQL databases handle OLAP workloads within a standardized data processing pipeline. Our experimental pipeline comprises Apache Spark for large-scale transformations, data cleansing, and schema normalization, diverse NoSQL databases as data stores, and a Business Intelligence tool for data analysis and visualization.

Evaluating NoSQL Databases for OLAP Workloads: A Benchmarking Study of MongoDB, Redis, Kudu and ArangoDB

TL;DR

This study investigates how NoSQL databases perform under OLAP workloads within a standardized data processing pipeline that includes HDFS, Spark, KoalaBench-derived inputs, and a BI layer. It benchmarks four NoSQL stores—MongoDB, Redis, ArangoDB, and Apache Kudu—plus PostgreSQL as a relational baseline, using Snowflake and Flat data models to reflect different schema designs. Across loading and five TPCH-like queries, the results show columnar storage (Apache Kudu) delivering the strongest performance, while other NoSQL options exhibit varied scalability and sometimes exponential growth in query times, especially with joins or complex aggregations. The findings support prioritizing columnar storage for scalable OLAP in NoSQL pipelines and point to future work integrating BI tools and expanding comparisons to additional columnar options.

Abstract

In the era of big data, conventional RDBMS models have become impractical for handling colossal workloads. Consequently, NoSQL databases have emerged as the preferred storage solutions for executing processing-intensive Online Analytical Processing (OLAP) tasks. Within the realm of NoSQL databases, various classifications exist based on their data storage mechanisms, making it challenging to select the most suitable one for a given OLAP workload. While each NoSQL database boasts distinct advantages, inherent scalability, adaptability to diverse data formats, and high data availability are universally recognized benefits crucial for managing OLAP workloads effectively. Existing research predominantly evaluates individual databases within custom data pipeline setups, lacking a standardized approach for comparative analysis across different databases to identify the optimal data pipeline for OLAP workloads. In this paper, we present our experimental insights into how various NoSQL databases handle OLAP workloads within a standardized data processing pipeline. Our experimental pipeline comprises Apache Spark for large-scale transformations, data cleansing, and schema normalization, diverse NoSQL databases as data stores, and a Business Intelligence tool for data analysis and visualization.
Paper Structure (26 sections, 8 figures, 1 table)

This paper contains 26 sections, 8 figures, 1 table.

Table of Contents

  1. Introduction
  2. Data Loading and Processing
  3. Data Store
  4. Related Work
  5. HDFS and Spark for OLAP
  6. Benchmarking for NoSQL-OLAP
  7. The Generic Pipeline
  8. Data Processing
  9. Snowflake Data Model
  10. Flat Data Model
  11. Data Store
  12. MongoDB
  13. ArangoDB
  14. Apache Kudu
  15. Redis
  16. ...and 11 more sections

Figures (8)

  • Figure 1: Schema of the dataset based off the Snow Logical Data Model generated by Koalabench
  • Figure 2: Schema of the dataset based off the Flat Logical Data Model generated by Koalabench
  • Figure 3: Dockerized experimental setup consisting of one master and three worker nodes
  • Figure 4: Query Execution time (in seconds) for PostgreSQL across the five datasets
  • Figure 5: Query Execution time (in seconds) for MongoDB across the five datasets
  • ...and 3 more figures