BinomialHash: A Constant Time, Minimal Memory Consistent Hash Algorithm

TL;DR

BinomialHash addresses the problem of distributing keys across a dynamic cluster with minimal reorganisation by achieving $O(1)$ lookup time and minimal memory usage. It uses a novel binary-tree-based mapping of the b-array into a hanging complete binary tree and performs a bounded number of rehash iterations over the enclosing tree and a smaller minor tree to ensure uniformity and monotonicity. The approach provides formal guarantees of $O(1)$ lookup time, minimal disruption during bucket changes, and a bounded imbalance that decays exponentially with the iteration bound $\omega$. Empirical benchmarks compare BinomialHash against PowerCH, FlipHash, and JumpBackHash, showing favorable lookup times and balance while using integer arithmetic and lacking floating-point operations.

Abstract

Consistent hashing is a technique for distributing data across a network of nodes in a way that minimizes reorganization when nodes join or leave the network. It is extensively applied in modern distributed systems as a fundamental mechanism for routing and data placement. Similarly, distributed storage systems rely on consistent hashing for scalable and fault-tolerant data partitioning. This paper introduces BinomialHash, a consistent hashing algorithm that executes in constant time and requires minimal memory. We provide a detailed explanation of the algorithm, present a pseudo-code implementation, and formally establish its strong theoretical guarantees. Finally, we compare its performance against state-of-the-art constant-time consistent hashing algorithms, demonstrating that our solution is both highly competitive and effective, while also validating the theoretical boundaries.