Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantization of Probability Distributions via Divide-and-Conquer: Convergence and Error Propagation under Distributional Arithmetic Operations

Bilgesu Arif Bilgin, Olof Hallqvist Elias, Michael Selby, Phillip Stanley-Marbell

TL;DR

This work introduces a divide-and-conquer quantization algorithm that builds a 2^n-atom discrete representation of any continuous distribution with finite mean, providing a simple and computable upper bound in Wasserstein-1 distance. It proves that, under a split-function framework, the error decays at quantifiable rates and remains stable under distributional arithmetic, with mean-preserving properties and tail-dependent behavior analyzed. The paper also provides comprehensive numerical experiments showing near-optimal convergence in many cases, demonstrated stability during arithmetic operations, and favorable comparisons to Monte Carlo methods. These results offer a deterministic, robust alternative to Monte Carlo for propagating uncertainty through arithmetic operations, with potential impact on probabilistic computing and stochastic numerical methods.

Abstract

This article studies a general divide-and-conquer algorithm for approximating continuous one-dimensional probability distributions with finite mean. The article presents a numerical study that compares pre-existing approximation schemes with a special focus on the stability of the discrete approximations when they undergo arithmetic operations. The main results are a simple upper bound of the approximation error in terms of the Wasserstein-1 distance that is valid for all continuous distributions with finite mean. In many use-cases, the studied method achieve optimal rate of convergence, and numerical experiments show that the algorithm is more stable than pre-existing approximation schemes in the context of arithmetic operations.

Quantization of Probability Distributions via Divide-and-Conquer: Convergence and Error Propagation under Distributional Arithmetic Operations

TL;DR

This work introduces a divide-and-conquer quantization algorithm that builds a 2^n-atom discrete representation of any continuous distribution with finite mean, providing a simple and computable upper bound in Wasserstein-1 distance. It proves that, under a split-function framework, the error decays at quantifiable rates and remains stable under distributional arithmetic, with mean-preserving properties and tail-dependent behavior analyzed. The paper also provides comprehensive numerical experiments showing near-optimal convergence in many cases, demonstrated stability during arithmetic operations, and favorable comparisons to Monte Carlo methods. These results offer a deterministic, robust alternative to Monte Carlo for propagating uncertainty through arithmetic operations, with potential impact on probabilistic computing and stochastic numerical methods.

Abstract

This article studies a general divide-and-conquer algorithm for approximating continuous one-dimensional probability distributions with finite mean. The article presents a numerical study that compares pre-existing approximation schemes with a special focus on the stability of the discrete approximations when they undergo arithmetic operations. The main results are a simple upper bound of the approximation error in terms of the Wasserstein-1 distance that is valid for all continuous distributions with finite mean. In many use-cases, the studied method achieve optimal rate of convergence, and numerical experiments show that the algorithm is more stable than pre-existing approximation schemes in the context of arithmetic operations.

Paper Structure

This paper contains 25 sections, 14 theorems, 156 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Contributions
  3. Theoretical main result
  4. Outline of the article
  5. Notation, definitions and prerequisites
  6. Wasserstein spaces
  7. Zador's Theorem
  8. Explicit optimal quantizer for Wasserstein-1
  9. Asymptotically optimal quantizer for Wasserstein-1
  10. A quantization algorithm for distributions
  11. Analysis of the algorithm
  12. Preliminary results
  13. A general upper bound for finite mean distributions
  14. Corollaries
  15. Examples
  16. ...and 10 more sections

Key Result

Theorem 1.1

Assume that $\mu$ is a continuous probability measure with finite mean and assume that $\mathrm{supp}\,(\mu) = {\mathbb{R}}_+$ and let $\mu^{(n)} = T(\mu,n)$. Let $\omega_{-1}= 0,\ \omega_0 = \int_{{\mathbb{R}}_+} t \mathrm{d} \mu(t)$ and define $\omega_{j+1} = {\mathbb{E}}[X | X \geq \omega_{j}], j

Figures (4)

  • Figure 1: Plots of Wasserstein-1 error vs representation size for mean-split, median-split, optimal and asymptotically optimal representations for the unit Gaussian (left) and unit exponential (right) distributions.
  • Figure 2: Log-log plots of Wasserstein-1 error vs representation size for mean-split, median-split, optimal and asymptotically optimal representations for the unit Gaussian (left) and unit exponential (right) distributions.
  • Figure 3: Plots of Wasserstein-1 error vs the number of additions for mean-split, median-split, optimal and asymptotically-optimal representations for the unit Gaussian (left) and unit exponential (right).
  • Figure 4: Plots of Wasserstein-1 error vs the number of multiplications for mean-split, median-split, optimal and asymptotically-optimal representations for the unit Gaussian (left) and unit exponential (right).

Theorems & Definitions (37)

  • Theorem 1.1
  • Theorem 1.2
  • Theorem 2.1: Theorem 6.2, p. 78 in MR1764176
  • Remark 2.1
  • Definition 3.1
  • Remark 3.1
  • Definition 3.2
  • Lemma 3.1
  • proof
  • Example 3.1
  • ...and 27 more