Optimal Static Fully Indexable Dictionaries

TL;DR

This work resolves a long-standing open question on static fully indexable dictionaries (FID) by achieving a succinct FID with near-optimal time in the regime where the universe size satisfies $U = n^{1+\Theta(1)}$. The authors develop a block-based framework that partitions keys and decomposes each key into low, mid, and high parts, employing inter-block structures, sparse high-part predecessors, and aB-trees to realize an overall worst-case query time of $O(t + \log\log n)$ with redundancy $R = n/(\log n / t)^{\Omega(t)}$. A key innovation is the Advanced FID that tightens the time to $O(t + \log\log n)$ by adaptively handling long low-part subsequences through a unified, non-recursive embedding of the basic FID, while preserving the same redundancy. The techniques yield broad implications for related problems, including improved predecessor and select dictionaries and a near-optimal partial-sum data structure for sequences of $\ell = \Theta(\log n)$-bit integers, all sharing the same core methodology. Overall, the results close the gap between upper and lower bounds for FIDs across a wide parameter range, establishing a tight, practical trade-off that advances space-efficient data structures for ranking, selection, and prefix-sum queries.

Abstract

Fully indexable dictionaries (FID) store sets of integer keys while supporting rank/select queries. They serve as basic building blocks in many succinct data structures. Despite the great importance of FIDs, no known FID is succinct with efficient query time when the universe size $U$ is a large polynomial in the number of keys $n$, which is the conventional parameter regime for dictionary problems. In this paper, we design an FID that uses $\log \binom{U}{n} + \frac{n}{(\log U / t)^{Ω(t)}}$ bits of space, and answers rank/select queries in $O(t + \log \log n)$ time in the worst case, for any parameter $1 \le t \le \log n / \log \log n$, provided $U = n^{1 + Θ(1)}$. This time-space trade-off matches known lower bounds for FIDs [Pǎtraşcu & Thorup STOC 2006; Pǎtraşcu & Viola SODA 2010] when $t \le \log^{0.99} n$. Our techniques also lead to efficient succinct data structures for the fundamental problem of maintaining $n$ integers each of $\ell = Θ(\log n)$ bits and supporting partial-sum queries, with a trade-off between $O(t)$ query time and $n\ell + n / (\log n / t)^{Ω(t)}$ bits of space. Prior to this work, no known data structure for the partial-sum problem achieves constant query time with $n \ell + o(n)$ bits of space usage.

Figures (1)

• Figure 1: Partitioning keys into three parts. We (a) divide the binary representations of the keys into low parts and mid-high parts, and further (b) take the difference sequence $(\delta_i)_{1 \le i \le B^t}$ of the mid-high parts and divide them into mid parts and high parts. There are $b \coloneqq \log (U/n) - h$ bits in the low part, $2h$ bits in the mid part, and $\log n - h$ bits in the high part.

Theorems & Definitions (15)

• Theorem 1.1
• Corollary 1.2
• Theorem 1.3
• Theorem 1.4
• Lemma 2.1: Natural generalization of patrascu2008succincter
• Lemma 2.2
• proof : Proof Sketch
• Theorem 3.1: Weak version of \ref{['thmt@@thmMainFID']}
• Theorem 4.1
• proof