Entropy Coding of Unordered Data Structures
Julius Kunze, Daniel Severo, Giulio Zani, Jan-Willem van de Meent, James Townsend
TL;DR
This work introduces shuffle coding, a general, lossless compression framework for unordered data structures by eliminating redundant order information with bits-back coding. It builds a formal bridge between ordered codecs and unordered objects using exchangeability and the orbit-stabilizer theorem, deriving the optimal rate $\log \frac{1}{Q(\tilde{f})} = \log \frac{1}{P(f)} - \log \frac{n!}{|\mathrm{Aut}(f)|}$ and implementing a practical UniformLCoset codec. Empirically, shuffle coding achieves state-of-the-art compression on a range of graphs, including molecular graphs, under Erdős-Rényi and Pólya urn models, while remaining compatible with simple ordered models and scalable to graphs with attributes. The work also discusses limitations (initial bits, computational complexity of canonization) and points to future directions such as approximations, bit-swap techniques, and integration with neural generative models to further improve rates on broader unordered data domains.
Abstract
We present shuffle coding, a general method for optimal compression of sequences of unordered objects using bits-back coding. Data structures that can be compressed using shuffle coding include multisets, graphs, hypergraphs, and others. We release an implementation that can easily be adapted to different data types and statistical models, and demonstrate that our implementation achieves state-of-the-art compression rates on a range of graph datasets including molecular data.