Optimizing Quotient Filters using Graveyard Hashing

TL;DR

Problem addressed: improve Quotient Filter performance at high load factors by mitigating primary clustering. Approach: Graveyard Filter uses tombstones and multiple redistribution policies to mitigate clustering, comparing variants against the Quotient Filter. Findings: some Graveyard variants outperform the Quotient Filter at load factors $0.75$ and above, with Graveyard Hashing offering the best high-load gains though workload sensitivity exists. Significance: a load-adaptive deployment of graveyard strategies can sustain higher throughput in AMQ applications, with further gains possible from parameter tuning and auxiliary structures.

Abstract

We aim to improve the performance of the Quotient Filter at high load factors. Our Graveyard Filter is a variation of the Quotient Filter which incorporates Graveyard Hashing, a technique that uses tombstones to counteract the effects of primary clustering. We summarize our implementation of the graveyard filter and detail approaches to redistributing tombstones. Evaluating these variations under conditions similar to the original quotient filter paper, we found the performance of the graveyard filter to be competitive for insertion and query operations, with certain redistribution schemes showing stronger performance at high load factors. We discuss potential further improvements, such as using the current load factor to determine the employed redistribution approach.