Random Cycle Coding: Lossless Compression of Cluster Assignments via Bits-Back Coding

TL;DR

The paper introduces Random Cycle Coding (RCC), a lossless compression method for cluster assignments that represents assignments via permutation cycles. By employing bits-back coding and Foata's canonicalization, RCC achieves Shannon-bound bit savings equal to $\log|\Pi|$ without training and with quasi-linear worst-case complexity in the largest cluster. The approach outperforms two variants of Random Order Coding (ROC) in both bit-rate savings and computational efficiency, and demonstrates practical impact on vector databases like FAISS by reducing labeling overhead and improving storage efficiency. The work provides a rigorous probabilistic model for RCC, experiments across large datasets, and a detailed complexity analysis, highlighting RCC's suitability for scalable, labeling-free clustering storage in similarity search systems.

Abstract

We present an optimal method for encoding cluster assignments of arbitrary data sets. Our method, Random Cycle Coding (RCC), encodes data sequentially and sends assignment information as cycles of the permutation defined by the order of encoded elements. RCC does not require any training and its worst-case complexity scales quasi-linearly with the size of the largest cluster. We characterize the achievable bit rates as a function of cluster sizes and number of elements, showing RCC consistently outperforms previous methods while requiring less compute and memory resources. Experiments show RCC can save up to 2 bytes per element when applied to vector databases, and removes the need for assigning integer ids to identify vectors, translating to savings of up to 70% in vector database systems for similarity search applications.

Figures (4)

• Figure 1: High-level description of our method, Random Cycle Coding (RCC). RCC encodes the clustering $(\tilde{\mathop{\mathrm{\mathcal{X}}}\nolimits}, \Pi)$ as cycles in the permutation $\sigma$ induced by the relative ordering of objects. Left: Indices represent the rankings of objects in $\tilde{\mathop{\mathrm{\mathcal{X}}}\nolimits}$ according to the total ordering of $\mathop{\mathrm{\mathcal{X}}}\nolimits$. Middle: One of two permutations will be randomly selected with bits-back coding to represent the clustering ($\sigma$, shown in cycle-notation). Then, Foata's Bijection is applied to yield $\sigma^\prime$, shown in line-notation, which is encoded into the ANS stack. Right: The final ANS stack containing $\sigma^\prime$ in line-notation.
• Figure 2: Median encoding plus decoding times, across $100$ runs, for Random Order Coding (ROC) severo2023compressing and our method Random Cycle Coding (RCC). The number of elements $n$ increases from left-to-right across plots. Clusters are fixed to have roughly the same size, $n/k$, mirroring vector database applications discussed in \ref{['sec:experiments-faiss']}. Reported times are representative of the amount of compute needed to sample the permutation in the bits-back step as data vectors are encoded with ANS using a uniform distribution.
• Figure 3: Maximum (left) and minimum (right) byte savings per element as a function of the number of clusters and elements. Savings are maximized when one cluster contains most elements and all others are singletons. The minimum is achieved if clusters have roughly the same number of elements. Two variants of Random Order Coding (ROC) severo2023compressing are shown (see \ref{['sec:related-work']}) with dashed lines. Random Cycle Coding (RCC) achieves higher savings than both variants of ROC while requiring less memory and computational resources.
• Figure 4: Pseudo-code for encoding (top) and decoding (bottom) a clustering of $n$ elements with our method, Random Cycle Coding (RCC). uniform_codec_decode samples an integer from state between $0$ and precision$-1$. append_to_cluster_and_sort inserts an element into a sorted list such that the resulting list is also sorted.

Theorems & Definitions (7)

• Example 2.1
• Lemma 2.2: Foata's Bijection foata1968netto
• Example 2.3
• Definition 3.1: Cycle-Cluster-Equivalence
• Example 3.2
• Definition 3.3: Foata's Canonicalization
• Example 3.4