Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lossless Embedding Compression via Spherical Coordinates

Han Xiao

TL;DR

This paper addresses the need for lossless compression of high-dimensional unit-norm embeddings without training. By transforming Cartesian coordinates to spherical angles on the hypersphere $S^{d-1}$, the method exploits exponent concentration near $\pi/2$, enabling effective entropy coding after a strategic byte-shuffling pipeline. Across 26 configurations (text, image, and multi-vector embeddings), the approach achieves approximately $1.5\\times$ compression with consistent, bit-exact reconstruction within float32 precision, and substantial practical gains for large-scale KV caches (e.g., ColBERT) with no training required. This work fills the gap between uncompressed storage and lossy quantization, offering a robust, fast, and exact solution for embedding storage and transmission, while also revealing favorable behavior for BF16 inputs.

Abstract

We present a lossless compression method for unit-norm embeddings that achieves 1.5$\times$ compression, 25\% better than the best prior method. The method exploits that spherical coordinates of high-dimensional unit vectors concentrate around $π/2$, causing IEEE 754 exponents to collapse to a single value and enabling entropy coding. Evaluation across 26 configurations spanning text, image, and multi-vector embeddings confirms consistent improvement. The method requires no training and is fully lossless within float32 precision.

Lossless Embedding Compression via Spherical Coordinates

TL;DR

This paper addresses the need for lossless compression of high-dimensional unit-norm embeddings without training. By transforming Cartesian coordinates to spherical angles on the hypersphere , the method exploits exponent concentration near , enabling effective entropy coding after a strategic byte-shuffling pipeline. Across 26 configurations (text, image, and multi-vector embeddings), the approach achieves approximately compression with consistent, bit-exact reconstruction within float32 precision, and substantial practical gains for large-scale KV caches (e.g., ColBERT) with no training required. This work fills the gap between uncompressed storage and lossy quantization, offering a robust, fast, and exact solution for embedding storage and transmission, while also revealing favorable behavior for BF16 inputs.

Abstract

We present a lossless compression method for unit-norm embeddings that achieves 1.5 compression, 25\% better than the best prior method. The method exploits that spherical coordinates of high-dimensional unit vectors concentrate around , causing IEEE 754 exponents to collapse to a single value and enabling entropy coding. Evaluation across 26 configurations spanning text, image, and multi-vector embeddings confirms consistent improvement. The method requires no training and is fully lossless within float32 precision.
Paper Structure (22 sections, 3 theorems, 1 equation, 2 figures, 9 tables, 1 algorithm)

This paper contains 22 sections, 3 theorems, 1 equation, 2 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contribution.
  3. Related Work
  4. Lossless Floating-Point Compression
  5. IEEE 754 Float32.
  6. Spherical and Polar Coordinate Methods
  7. Spherical Coordinates.
  8. Lossy Vector Compression
  9. Method
  10. Evaluation
  11. Experimental Setup
  12. Main Results
  13. Conclusion
  14. Python Implementation
  15. Compression Speed
  16. ...and 7 more sections

Key Result

Theorem 1

For any value $\theta \in [a, b]$ with $0 < a \leq b$, the IEEE 754 exponent satisfies $\mathcal{E}(\theta) \in \{e_{\min}, \ldots, e_{\max}\}$ where $e_{\min} = \lfloor \log_2 a \rfloor + 127$ and $e_{\max} = \lfloor \log_2 b \rfloor + 127$. The number of distinct exponents is at most $\lfloor \log

Figures (2)

  • Figure 1: Compression pipeline. Cartesian coordinates span diverse magnitudes with 20 to 40 different exponents, shown in varied colors. The spherical transform produces angles concentrated around $\pi/2 \approx 1.57$, collapsing nearly all exponents to 127, shown in uniform color. Transpose groups same-position angles across vectors, byte shuffle separates exponent bytes, and zstd compresses the low-entropy exponent stream.
  • Figure 2: IEEE 754 exponent distribution for https://huggingface.co/jinaai/jina-embeddings-v4 (2048d). (a) Cartesian coordinates span 49 exponent values; (b) spherical angles concentrate around exponent 127 with 99.7% frequency.

Theorems & Definitions (4)

  • Theorem 1: Exponent Bound for Bounded Intervals
  • proof
  • Corollary 2: Spherical Angle Exponents
  • Corollary 3: Entropy Upper Bound