GriDB: Scaling Blockchain Database via Sharding and Off-Chain Cross-Shard Mechanism

TL;DR

Addressing blockchain scalability, the paper tackles the high cost of cross-shard database services by moving heavy data exchanges off-chain. It introduces GriDB, a relational sharding blockchain database that uses a delegation-based off-chain cross-shard mechanism and authenticated data structures (ADS) including Verifiable Set Operations (VSO) and Merkle trees to produce succinct proofs verifiable on-chain. It also proposes an off-chain live inter-shard migration scheme with a Bloom-filter aided query optimization and a greedy load-balancing scheduler. Experimental evaluation on a 16-machine testbed demonstrates linear throughput with shard count, orders-of-magnitude faster cross-shard query proofs than SNARK-based approaches, and significant migration-time reductions. This work enables scalable, SQL-capable blockchain databases by reducing on-chain consensus burdens while preserving correctness, completeness, freshness, and availability of cross-shard services.

Abstract

Blockchain databases have attracted widespread attention but suffer from poor scalability due to underlying non-scalable blockchains. While blockchain sharding is necessary for a scalable blockchain database, it poses a new challenge named on-chain cross-shard database services. Each cross-shard database service (e.g., cross-shard queries or inter-shard load balancing) involves massive cross-shard data exchanges, while the existing cross-shard mechanisms need to process each cross-shard data exchange via the consensus of all nodes in the related shards (i.e., on-chain) to resist a Byzantine environment of blockchain, which eliminates sharding benefits. To tackle the challenge, this paper presents GriDB, the first scalable blockchain database, by designing a novel off-chain cross-shard mechanism for efficient cross-shard database services. Borrowing the idea of off-chain payments, GriDB delegates massive cross-shard data exchange to a few nodes, each of which is randomly picked from a different shard. Considering the Byzantine environment, the untrusted delegates cooperate to generate succinct proof for cross-shard data exchanges, while the consensus is only responsible for the low-cost proof verification. However, different from payments, the database services' verification has more requirements (e.g., completeness, correctness, freshness, and availability); thus, we introduce several new authenticated data structures (ADS). Particularly, we utilize consensus to extend the threat model and reduce the complexity of traditional accumulator-based ADS for verifiable cross-shard queries with a rich set of relational operators. Moreover, we study the necessity of inter-shard load balancing for a scalable blockchain database and design an off-chain and live approach for both efficiency and availability during balancing.

Figures (9)

• Figure 1: (a) Illustration for sharding blockchain database, which requires two new functions, i.e., data aggregation for query and workload balancing for management. (b) Transaction throughput of non-sharding and sharding blockchain databases. (cxq. represents cross-shard query.)
• Figure 2: System overview for GriDB.
• Figure 3: Example for ADS proof generation of two tables distributed in Shard A and B, respectively. ($\sigma$ is an operator to select rows from a relation and $\Join$ is an operator to join tables based on a specified column.)
• Figure 4: Overview of off-chain live migration. A solid line with arrowhead represents a cross-shard transaction and a dotted line with arrowhead represents an off-chain cross-shard communication.
• Figure 5: Transaction throughput for GriDB, the on-chain sharding blockchain database, and the non-sharding blockchain database (cx means cross-shard ratio.)

Theorems & Definitions (4)

• Definition 1
• Theorem 1
• Definition 2
• Theorem 2