The Poisson tensor completion parametric estimator

Abstract

We introduce the Poisson tensor completion (PTC) estimator that exploits inter-sample relationships to compute a low-rank Poisson tensor decomposition of the frequency histogram for samples of a multivariate distribution. Our crucial observation is that the histogram bins are an instance of a space partitioning of counts and thus can be identified with a spatial non-homogeneous Poisson process. The Poisson tensor decomposition leads to a completion of the mean measure over all bins -- including those containing few to no samples -- and leads to our proposed estimator. A Poisson tensor decomposition models the underlying distribution of the count data and guarantees non-negative estimated values obviating the need for additional constraints to ensure non-negativity. Furthermore, we demonstrate that our PTC estimator is a substantial improvement over standard histogram-based estimators for sub-Gaussian probability distributions because of the concentration of norm phenomenon.

Figures (10)

• Figure 1: Comparing estimates and bins used from 25 trials for dimension $6$ distributions. The histogram is constructed by placing $s=2500$ samples from the distributions into bins of width $c\, s^{-\frac{1}{8}}$ in each dimension, for different values of $c$. Here, the tensor-based approximation uses rank $5$ and the same bins as the histogram-based approximation.
• Figure 2: The proportion of nonempty bins for the histograms used to estimate entropy in Figure \ref{['fig:bin-schemes']}. The distributions are dimension $d=6$, and the histograms use bins of width $c\cdot(2500)^{-\frac{1}{8}}$ in each dimension, for different values of $c$. The number of bins decreases with increasing $c$.
• Figure 3: Error in estimated entropy of a five dimensional uniform distribution over (a) $[0,1]^5$ or (b) $[0,e^2]^5$ with independent dimensions. Estimates use a histogram directly, the tensor (PTC) method, or the $k$-NN method. The results shown are for 25 trials using the $k \in\{1,2,3,\dots,10,25,50,100,200\}$ or rank not exceeding five leading to the smallest error. The histogram uses bins of width $3.5s^{-1/7}$, where $s$ is the number of samples.
• Figure 4: Error in estimated entropy using a histogram directly, the tensor (PTC) method, or using the $k$-NN method. The results shown are for dimension 5, over 25 trials using the $k \in\{1,2,3,\dots,10,25,50,100,200\}$ or rank $\leq 5$ leading to the smallest error. The histogram uses bins of width $3.5s^{-1/7}$, where $s$ is the number of samples. The distributions shown are (a) Normal with independent dimensions (b) Normal with correlation between dimensions, and (c) $t$ with one degree of freedom and independent dimensions (equivalent to a Cauchy distribution).
• Figure 5: Estimated entropy for Gaussian mixtures of dimension $2,3,\dots,6$ with different ranks and different numbers of components with equidistant modes. Dotted lines: tensor estimates from $25$ trials of $s=2500$ samples from the distribution. Solid lines: average histogram estimate in $25$ trials with $s=1000000$ samples. The tensors use 20 bins along each dimension, and the histograms use bins of width $c\cdot s^{-\frac{1}{\text{dim}+2}}$ in each dimension.