Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

QMDB: Quick Merkle Database

Isaac Zhang, Ryan Zarick, Daniel Wong, Thomas Kim, Bryan Pellegrino, Mignon Li, Kelvin Wong

TL;DR

QMDB introduces a unified, SSD-optimized authenticated data store that merges key-value storage with Merkle-tree state, enabling fully in-memory Merkleization and minimal disk I/O. By packing updates into fixed-depth twigs and storing state changes in an append-only log, QMDB achieves up to 6x throughput over RocksDB and 8x over NOMT, while reducing DRAM needs to a few bytes per entry and supporting up to hundreds of billions of entries on capable hardware. The design supports historical state proofs, three-stage pipelining, and sharding to scale across commodity to enterprise hardware, and demonstrates practical performance on workloads exceeding Ethereum’s 2024 state by orders of magnitude. These advances lower barriers to participation, improve validator scalability, and enable new verifiable blockchain applications with strong proof guarantees and efficient state querying.

Abstract

Quick Merkle Database (QMDB) addresses longstanding bottlenecks in blockchain state management by integrating key-value (KV) and Merkle tree storage into a single unified architecture. QMDB delivers a significant throughput improvement over existing architectures, achieving up to 6X over the widely used RocksDB and 8X over NOMT, a leading verifiable database. Its novel append-only twig-based design enables one SSD read per state access, O(1) IOs for updates, and in-memory Merkleization on a memory footprint as small as 2.3 bytes per entry, enabling it to run on even modest consumer-grade PCs. QMDB scales seamlessly across both commodity and enterprise hardware, achieving up to 2.28 million state updates per second. This performance enables support for 1 million token transfers per second (TPS), marking QMDB as the first solution achieving such a milestone. QMDB has been benchmarked with workloads exceeding 15 billion entries (10X Ethereum's 2024 state) and has proven the capacity to scale to 280 billion entries on a single server. Furthermore, QMDB introduces historical proofs, unlocking the ability to query its blockchain's historical state at the latest block. QMDB not only meets the demands of current blockchains but also provides a robust foundation for building scalable, efficient, and verifiable decentralized applications across diverse use cases.

QMDB: Quick Merkle Database

TL;DR

QMDB introduces a unified, SSD-optimized authenticated data store that merges key-value storage with Merkle-tree state, enabling fully in-memory Merkleization and minimal disk I/O. By packing updates into fixed-depth twigs and storing state changes in an append-only log, QMDB achieves up to 6x throughput over RocksDB and 8x over NOMT, while reducing DRAM needs to a few bytes per entry and supporting up to hundreds of billions of entries on capable hardware. The design supports historical state proofs, three-stage pipelining, and sharding to scale across commodity to enterprise hardware, and demonstrates practical performance on workloads exceeding Ethereum’s 2024 state by orders of magnitude. These advances lower barriers to participation, improve validator scalability, and enable new verifiable blockchain applications with strong proof guarantees and efficient state querying.

Abstract

Quick Merkle Database (QMDB) addresses longstanding bottlenecks in blockchain state management by integrating key-value (KV) and Merkle tree storage into a single unified architecture. QMDB delivers a significant throughput improvement over existing architectures, achieving up to 6X over the widely used RocksDB and 8X over NOMT, a leading verifiable database. Its novel append-only twig-based design enables one SSD read per state access, O(1) IOs for updates, and in-memory Merkleization on a memory footprint as small as 2.3 bytes per entry, enabling it to run on even modest consumer-grade PCs. QMDB scales seamlessly across both commodity and enterprise hardware, achieving up to 2.28 million state updates per second. This performance enables support for 1 million token transfers per second (TPS), marking QMDB as the first solution achieving such a milestone. QMDB has been benchmarked with workloads exceeding 15 billion entries (10X Ethereum's 2024 state) and has proven the capacity to scale to 280 billion entries on a single server. Furthermore, QMDB introduces historical proofs, unlocking the ability to query its blockchain's historical state at the latest block. QMDB not only meets the demands of current blockchains but also provides a robust foundation for building scalable, efficient, and verifiable decentralized applications across diverse use cases.
Paper Structure (16 sections, 5 equations, 3 figures, 4 tables)

This paper contains 16 sections, 5 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. QMDB Design
  4. Entries and Twigs
  5. Indexer
  6. CRUD interface
  7. Proofs
  8. Parallelization
  9. Evaluation
  10. 6X more updates/s than key-value DBs
  11. Up to 8X throughput vs state-of-the-art
  12. Reaching 2M updates per second
  13. Scaling up and down
  14. Discussion
  15. Conclusion
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Entries are inserted sequentially into the leaves of the Fresh twig, and all leaves have the same depth. The twig eventually transitions into the Full state. As Entries are deleted, Full twigs become Inactive, then transition to Pruned. Upper nodes are recursively pruned after both of their children are pruned.
  • Figure 2: QMDB prefetches data (prefetcher), performs the state transition (updater), then commits the updated state to the Merkle tree and persistent storage (committer).
  • Figure 3: QMDB shows a $6 \times$ increase in throughput over RocksDB. QMDB is able to do 601K updates/sec with 6 billion entries and demonstrates superior performance across all operation types. These results were obtained on an AWS c7gd.metal instance with 2 SSDs and 64 vCPUs.