Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rejection-Sampled Universal Quantization for Smaller Quantization Errors

Chih Wei Ling, Cheuk Ting Li

TL;DR

This work addresses entropy-constrained quantization by introducing rejection-sampled universal quantizers (RSUQ) that shape the quantization error distribution to be uniform over a prescribed region via rejection sampling on top of universal quantization.RSUQ achieves strictly smaller maximum error than all known lattice quantizers with the same entropy for dimensions $n=5$ to $48$, and smaller mean-squared error than lattice quantizers for $n=35$ to $47$ in the high-resolution limit, while preserving an input-independent error distribution such as uniform over a ball.The paper also develops nonuniform-input (LRSUQ) and nonuniform-error constructions, establishing universal-quantization properties and a one-shot channel-simulation perspective with constant-gap bounds relative to rate-distortion limits, and provides practical implications using simple lattices like $oldsymbol{Z}^n$.Together, these results advance high-dimensional, entropy-constrained quantization with simpler, lattice-insensitive design and exact additive-noise channel simulation, with potential applications in machine learning and privacy-preserving technologies.

Abstract

We construct a randomized vector quantizer which has a smaller maximum error compared to all known lattice quantizers with the same entropy for dimensions 5, 6, ..., 48, and also has a smaller mean squared error compared to known lattice quantizers with the same entropy for dimensions 35, ..., 47, in the high resolution limit. Moreover, our randomized quantizer has a desirable property that the quantization error is always uniform over the ball and independent of the input. Our construction is based on applying rejection sampling on universal quantization, which allows us to shape the error distribution to be any continuous distribution, not only uniform distributions over basic cells of a lattice as in conventional dithered quantization. We also characterize the high SNR limit of one-shot channel simulation for any additive noise channel under a mild assumption (e.g., the AWGN channel), up to an additive constant of 1.45 bits.

Rejection-Sampled Universal Quantization for Smaller Quantization Errors

TL;DR

This work addresses entropy-constrained quantization by introducing rejection-sampled universal quantizers (RSUQ) that shape the quantization error distribution to be uniform over a prescribed region via rejection sampling on top of universal quantization.RSUQ achieves strictly smaller maximum error than all known lattice quantizers with the same entropy for dimensions $n=5$ to $48$, and smaller mean-squared error than lattice quantizers for $n=35$ to $47$ in the high-resolution limit, while preserving an input-independent error distribution such as uniform over a ball.The paper also develops nonuniform-input (LRSUQ) and nonuniform-error constructions, establishing universal-quantization properties and a one-shot channel-simulation perspective with constant-gap bounds relative to rate-distortion limits, and provides practical implications using simple lattices like $oldsymbol{Z}^n$.Together, these results advance high-dimensional, entropy-constrained quantization with simpler, lattice-insensitive design and exact additive-noise channel simulation, with potential applications in machine learning and privacy-preserving technologies.

Abstract

We construct a randomized vector quantizer which has a smaller maximum error compared to all known lattice quantizers with the same entropy for dimensions 5, 6, ..., 48, and also has a smaller mean squared error compared to known lattice quantizers with the same entropy for dimensions 35, ..., 47, in the high resolution limit. Moreover, our randomized quantizer has a desirable property that the quantization error is always uniform over the ball and independent of the input. Our construction is based on applying rejection sampling on universal quantization, which allows us to shape the error distribution to be any continuous distribution, not only uniform distributions over basic cells of a lattice as in conventional dithered quantization. We also characterize the high SNR limit of one-shot channel simulation for any additive noise channel under a mild assumption (e.g., the AWGN channel), up to an additive constant of 1.45 bits.
Paper Structure (29 sections, 14 theorems, 106 equations, 4 figures, 1 table, 4 algorithms)

This paper contains 29 sections, 14 theorems, 106 equations, 4 figures, 1 table, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Randomized Quantizers
  4. Lattice Quantizers
  5. Rejection-Sampled Universal Quantizers
  6. RSUQ with Error Uniform over a General Set
  7. RSUQ with Error Uniform over an $n$-Ball
  8. Time Complexity of RSUQ
  9. RSUQ with Nonuniform Input Distribution and Universal Quantization
  10. Nonuniform Error Distribution
  11. Excess information of LRSUQ
  12. Conclusion and Discussions
  13. Acknowledgements
  14. Proof of Proposition \ref{['prop:lb']}
  15. Proof of Proposition \ref{['prop:error_dist_unif']}
  16. ...and 14 more sections

Key Result

Proposition 3

For any randomized quantizer $(P_{S},Q)$, we have the following lower bounds on the normalized entropy:

Figures (4)

  • Figure 1: RSUQ with $\mathbf{G} = \mathbf{I}_2$ (the $2 \times 2$ identity matrix), yielding the two-dimensional integer lattice $\mathbb{Z}^2$ with the basic cell $\mathcal{P}=\left(-1/2,1/2\right]^2$ (the Voronoi cell), and $\mathcal{A} = (1/2)\cdot B^2$ (a two-dimensional ball with radius $1/2$). Fix some $\mathbf{x} \in \mathbb{R}^2$. For $i=1,2,\ldots$, generate $\mathbf{V}_i \in \left(-1/2,1/2\right]^2$ and find the unique $\mathbf{M} \in \mathbb{Z}^2$ such that $\mathbf{M}+\mathbf{V}_i \in (\mathbb{Z}^2+\mathbf{V}_i) \cap (\mathbf{x}+\left(-1/2,1/2\right]^2)$. The first and second iterations (left and middle figures) are rejected because $\mathbf{M}+\mathbf{V}_i-\mathbf{x} \notin (1/2)\cdot B^2$, for $i = 1, 2$, and the third iteration (right figure) is accepted because $\mathbf{M}+\mathbf{V}_3-\mathbf{x} \in (1/2)\cdot B^2$.
  • Figure 2: Log-scale plot of the redundancy w.r.t. maximum error \ref{['eq:red_maxerror']} (left figure) and the Zador redundancy w.r.t. MSE \ref{['eq:red_mse_zador']} (right figure) of the lattice quantizer with the best known covering radius $\overline{\lambda}(\mathbf{G})$ on the left, and with the best known NSM $G_n(\mathcal{V})$ on the right (red line), the shift-periodic quantizer (green line), the RSUQ with error distribution $\mathrm{Unif}(B^{n})$ constructed using the best known sphere packing lattice (blue line), the RSUQ constructed using an arbitrary lattice (black line), Zador's upper bound \ref{['eq:zador_ub']} (violet line), and Ordentlich's upper bound \ref{['eq:or_bound']} (orange line).
  • Figure 3: Comparison of the lower bound on $\Phi(Q)$ for any randomized quantizer simulating the Gaussian channel (blue curve), $\Phi(Q)$ of LRSUQ (orange curve), $\Phi(Q)$ of LSPQ introduced in ling2023vector (green curve), and excess information of RDQ proposed in KobusRotated2024 (red curve), across various dimensions.
  • Figure 4: Log-scale plot of the Zador redundancy w.r.t. MSE \ref{['eq:lattice_mse_zador']} of the lattice quantizer with the NSM $G_n(\mathcal{V})$ from agrell2023best (red line), from Kudryashov2010LowComplexLattice (cyan diamond), from Lyu2022BetterLatticeQuan (cyan cross), and from Agrell2025OptLatticeQuan (cyan star), the shift-periodic quantizer (green line), the RSUQ with error distribution $\mathrm{Unif}(B^{n})$ constructed using the best known sphere packing lattice (blue line), the RSUQ constructed using an arbitrary lattice (black line), Zador's upper bound \ref{['eq:zador_ub']} (violet line), and Ordentlich's upper bound \ref{['eq:or_bound']} (orange line).

Theorems & Definitions (20)

  • Definition 1
  • Definition 2
  • Proposition 3: Rate-distortion lower bounds
  • Definition 4
  • Proposition 5
  • Theorem 6
  • Corollary 7
  • Theorem 8
  • Corollary 9
  • Proposition 10
  • ...and 10 more