Techniques for Authenticating Quantile Digests

TL;DR

This work tackles authenticating q-digests, compact data structures for distributions, in untrusted networks. It identifies correctness issues with the original compression, proposes two robust compression methods (RecursiveCompression and IterativeCompression) with formal correctness proofs and complexity analyses, and introduces two authentication approaches: Whole Digest Authentication (WDA) and KVC-Authenticated Queries (KVC-QA). The paper also examines the trade-offs between these methods in terms of space, time, and privacy, and outlines practical enhancements, such as refined size bounds, partial cumulative digests, and homomorphic KVCs for more scalable authentication. Collectively, these contributions provide a foundation for secure, efficient, and privacy-aware querying of compact data structures in distributed settings, with clear avenues for future improvements and broader applications.

Abstract

We investigate two possible techniques to authenticate the q-digest data structure, along with a worst-case study of the computational complexity both in time and space of the proposed solutions, and considerations on the feasibility of the presented approaches in real-world scenarios. We conclude the discussion by presenting some considerations on the information complexity of the queries in the two proposed approaches, and by presenting some interesting ideas that could be the subject of future studies on the topic.

Figures (5)

• Figure 1: Compression during merging of two q-digests may lead to errors. Each node is labelled with its index next to it. The two starting digests have both $k=4$, and respectively $n=38$ and $n=36$. The resulting digest has $n=74$, and consequently $\left\lfloor \frac{n}{k} \right\rfloor = 18$.
• Figure 2: Sample q-digest with $\sigma = 4, k = 2, n = 8$ for which the summation term is bigger than $3n$. Indeed: $\sum_{b \in Q} \nabla_Q(b) = 4 \times 6 + 1 = 25 \quad\nless\quad 24 = 3 \times 8 = 3n$.
• Figure 3: Sample q-digest with $\sigma = 64, n = 22, k = 4$ that has more than $3k$ nodes, despite the fact that both properties in \ref{['def:qdigest']} are satisfied. Indeed: $\left\lfloor \frac{n}{k} \right\rfloor = \left\lfloor \frac{22}{4} \right\rfloor = 5$, and $\left|Q\right| = 13 \quad\nless\quad 12 = 3k$.
• Figure 4: Q-Digest ($n = 15$, $k = 5$, $\sigma = 8$) on which the query is being executed.
• Figure :

Theorems & Definitions (20)

• Definition 1: $\nabla$ Function
• Definition 2: Q-Digest
• Definition 3: Q-Digest sum
• Definition 4: Q-Digest merge
• Theorem 1: Construction Invariant
• proof
• Corollary 1
• Example 1
• Theorem 2: RecursiveCompress Correctness
• proof