Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Conditional Measure Quantization

Gabriel Turinici

TL;DR

This work tackles the problem of quantizing a conditional probability law $\\mu_x=\\mu(\\cdot|X=x)$ by representing it as a finite Dirac mixture $\\delta^eta_{\\mathbf{y}(x)}$, using a Huber-energy distance $d$ to measure proximity. It introduces Deep Conditional Measure Quantization (DCMQ), a deep-net–driven approach that maps conditioning $x$ to a set of quantization points $\\mathbf{y}(x)$ to minimize $d(\\delta^eta_{\\mathbf{y}(x)},\\mu_x)^2$, with a conditional-sampling variant and a joint-sampling variant for MNIST restoration. Theoretical results establish the existence of measurable (and, in 1D, continuous) selections of conditional quantizers, while experiments on 2D Gaussian cases demonstrate convergence and conditional-tracking behavior, and MNIST restoration showcases practical applicability. Overall, the paper provides a general, interpolation-friendly framework for conditional distribution quantization with potential impact in statistics, signal processing, and related domains.

Abstract

Quantization of a probability measure means representing it with a finite set of Dirac masses that approximates the input distribution well enough (in some metric space of probability measures). Various methods exists to do so, but the situation of quantizing a conditional law has been less explored. We propose a method, called DCMQ, involving a Huber-energy kernel-based approach coupled with a deep neural network architecture. The method is tested on several examples and obtains promising results.

Deep Conditional Measure Quantization

TL;DR

This work tackles the problem of quantizing a conditional probability law by representing it as a finite Dirac mixture , using a Huber-energy distance to measure proximity. It introduces Deep Conditional Measure Quantization (DCMQ), a deep-net–driven approach that maps conditioning to a set of quantization points to minimize , with a conditional-sampling variant and a joint-sampling variant for MNIST restoration. Theoretical results establish the existence of measurable (and, in 1D, continuous) selections of conditional quantizers, while experiments on 2D Gaussian cases demonstrate convergence and conditional-tracking behavior, and MNIST restoration showcases practical applicability. Overall, the paper provides a general, interpolation-friendly framework for conditional distribution quantization with potential impact in statistics, signal processing, and related domains.

Abstract

Quantization of a probability measure means representing it with a finite set of Dirac masses that approximates the input distribution well enough (in some metric space of probability measures). Various methods exists to do so, but the situation of quantizing a conditional law has been less explored. We propose a method, called DCMQ, involving a Huber-energy kernel-based approach coupled with a deep neural network architecture. The method is tested on several examples and obtains promising results.
Paper Structure (14 sections, 2 theorems, 7 equations, 7 figures, 2 algorithms)

This paper contains 14 sections, 2 theorems, 7 equations, 7 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Conditional measure quantization : motivation
  3. Brief literature review on measure quantization, conditional sampling and conditional quantization
  4. Interpolation by deep neural networks
  5. The deep neural network conditional quantization method
  6. Setting and notations
  7. Existence of a measurable conditional quantization
  8. Existence of a continuous conditional quantization in 1D
  9. The Deep Conditional Quantization algorithm (DCMQ) : conditional sampling version
  10. Quantization of 2D Gaussian conditioned on its mean
  11. Quantization of 2D Gaussian : the multiplicative case
  12. Quantization of a 1D Gaussian mixture crossing
  13. The Deep Conditional Quantization through joint sampling : MNIST restauration
  14. Concluding remarks

Key Result

Proposition 1

Suppose that the bi-variate distribution $\mu$ is such that for any $x\in \mathcal{X}$ the distribution $\mu_x$ has finite $r$-th order moment. Then there exists a measurable function $\mathbf{y}^{opt} : \mathcal{X} \to \mathcal{Y}^Q$ such that $\mathbf{y}^{opt}(x)$ satisfies equation eq:defintion_q

Figures (7)

  • Figure 1: Conditional quantization with $Q=10$ points for the test in section \ref{['sec:numerics_2Dgaussian_additive']}. Left image: Convergence of the loss function. Right images : Five points $x_1, ..., x_5\in\mathcal{X} = \mathbb{R}^2$ are sampled from $\mu^X$ (plotted as blue triangles); the DNN (after training) is asked to quantize the conditional distribution $\mu_{x_b}$ for each $b\le 5$ (red stars). Recall that $\mu_{x_b}$ is a Gaussian shifted by $x_b$. The quantization points follow precisely the indicated mean.
  • Figure 2: Conditional quantization with $Q=10$ points for the test in section \ref{['sec:numerics_2Dgaussian_additive']}. The conditional quantization points $x\in \mathbb{R}^2 \mapsto \mathbf{y}^{dcmq}(x)=(\mathbf{y}_1^{dcmq}(x),...,\mathbf{y}_{10}^{dcmq}(x)) \in (\mathbb{R}^2)^{10}$ are plotted as functions of $x$; each column is a dimension of $\mathbf{y}^{dcmq}(x)$ (each row a dimension of the value itself); for graphical convenience we only plot the first $3$ quantized functions i.e. $\mathbf{y}_1^{dcmq}(x)$ (first column), $\mathbf{y}_2^{dcmq}(x)$ (second column), $\mathbf{y}_3^{dcmq}(x)$ (third column). The functions appear smooth and move synchronously, which is a suitable property of the conditional quantization, see section \ref{['sec:theory']}.
  • Figure 3: Conditional quantization with $Q=10$ points for the test in section \ref{['sec:numerics_2Dgaussian_std']}. Left image: Convergence of the loss function. Right images : Five points $x_1, ..., x_5\in\mathcal{X} = \mathbb{R}^2$ are sampled from $\mu^X$(blue triangles); the DNN (after training) is asked to quantize $\mu_{x_b}$ for each $b\le 5$ (red stars). Here $\mu_{x_b}$ is a Gaussian multiplied in each direction by $x_b$. So, for instance when a point $x_b$ has both component values large, the corresponding quantization will look like the quantization of a bi-variate normal. But when $x_b$ is close to some axis, the quantization will act on a very elliptical form distribution because one of the Gaussian is multiplied by a small constant. This expected behavior is reproduced well by the converged DNN.
  • Figure 4: Conditional quantization with $Q=10$ points for the test in section \ref{['sec:numerics_2Dgaussian_std']}. We plot the conditional quantization functions $\mathbf{y}_1^{dcmq}(x), \mathbf{y}_2^{dcmq}(x), \mathbf{y}_3^{dcmq}(x)$ as in figure \ref{['fig:several_quantizations10_conv']}.
  • Figure 5: Conditional quantization with $Q=4$ points for the test described in section \ref{['sec:numerics_1Dgaussian_mixture']}. Top image: Convergence of the loss function. Bottom image : The joint density of $(X,Y)$ is plotted in background (red are low values, blue are high values). The green dots indicate the quantized values. Note that each of the two parts of the mixture is assigned two quantization points that move along the mean.
  • ...and 2 more figures

Theorems & Definitions (9)

  • Proposition 1
  • Remark 2
  • proof
  • Remark 3
  • Proposition 4
  • proof
  • Remark 5
  • Remark 6
  • Remark 7