Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Simple Method for PMF Estimation on Large Supports

Alex Shtoff

TL;DR

This paper addresses nonparametric PMF estimation on large discrete supports with heavy-tailed, multi-modal structure. It introduces a simple, fast method that treats the empirical PMF as a signal on a line graph and performs a data-adaptive spectral projection by solving an eigenproblem for the Schrödinger-type operator ${\bm{H}}={\bm{L}}-\mathrm{diag}({\bm{p}})$. By projecting onto the subspace spanned by the $k$ smallest eigenvectors and normalizing the positive part, the method yields a smooth, coarse-structure-preserving PMF; $k$ is selected via a data-driven risk criterion inspired by orthogonal-series estimation. Empirically, the approach is competitive with logspline and often outperforms Gaussian KDE in heavy-tailed settings, with robust performance across sample sizes and scalable computation on large supports. The work highlights a practical, parameter-light alternative for automated visualization and feature engineering at scale, while noting limitations such as abrupt discontinuities and discussing future directions for basis design and alternative formulation.

Abstract

We study nonparametric estimation of a probability mass function (PMF) on a large discrete support, where the PMF is multi-modal and heavy-tailed. The core idea is to treat the empirical PMF as a signal on a line graph and apply a data-dependent low-pass filter. Concretely, we form a symmetric tri-diagonal operator, the path graph Laplacian perturbed with a diagonal matrix built from the empirical PMF, then compute the eigenvectors, corresponding to the smallest feq eigenvalues. Projecting the empirical PMF onto this low dimensional subspace produces a smooth, multi-modal estimate that preserves coarse structure while suppressing noise. A light post-processing step of clipping and re-normalizing yields a valid PMF. Because we compute the eigenpairs of a symmetric tridiagonal matrix, the computation is reliable and runs time and memory proportional to the support times the dimension of the desired low-dimensional supspace. We also provide a practical, data-driven rule for selecting the dimension based on an orthogonal-series risk estimate, so the method "just works" with minimal tuning. On synthetic and real heavy-tailed examples, the approach preserves coarse structure while suppressing sampling noise, compares favorably to logspline and Gaussian-KDE baselines in the intended regimes. However, it has known failure modes (e.g., abrupt discontinuities). The method is short to implement, robust across sample sizes, and suitable for automated pipelines and exploratory analysis at scale because of its reliability and speed.

A Simple Method for PMF Estimation on Large Supports

TL;DR

This paper addresses nonparametric PMF estimation on large discrete supports with heavy-tailed, multi-modal structure. It introduces a simple, fast method that treats the empirical PMF as a signal on a line graph and performs a data-adaptive spectral projection by solving an eigenproblem for the Schrödinger-type operator . By projecting onto the subspace spanned by the smallest eigenvectors and normalizing the positive part, the method yields a smooth, coarse-structure-preserving PMF; is selected via a data-driven risk criterion inspired by orthogonal-series estimation. Empirically, the approach is competitive with logspline and often outperforms Gaussian KDE in heavy-tailed settings, with robust performance across sample sizes and scalable computation on large supports. The work highlights a practical, parameter-light alternative for automated visualization and feature engineering at scale, while noting limitations such as abrupt discontinuities and discussing future directions for basis design and alternative formulation.

Abstract

We study nonparametric estimation of a probability mass function (PMF) on a large discrete support, where the PMF is multi-modal and heavy-tailed. The core idea is to treat the empirical PMF as a signal on a line graph and apply a data-dependent low-pass filter. Concretely, we form a symmetric tri-diagonal operator, the path graph Laplacian perturbed with a diagonal matrix built from the empirical PMF, then compute the eigenvectors, corresponding to the smallest feq eigenvalues. Projecting the empirical PMF onto this low dimensional subspace produces a smooth, multi-modal estimate that preserves coarse structure while suppressing noise. A light post-processing step of clipping and re-normalizing yields a valid PMF. Because we compute the eigenpairs of a symmetric tridiagonal matrix, the computation is reliable and runs time and memory proportional to the support times the dimension of the desired low-dimensional supspace. We also provide a practical, data-driven rule for selecting the dimension based on an orthogonal-series risk estimate, so the method "just works" with minimal tuning. On synthetic and real heavy-tailed examples, the approach preserves coarse structure while suppressing sampling noise, compares favorably to logspline and Gaussian-KDE baselines in the intended regimes. However, it has known failure modes (e.g., abrupt discontinuities). The method is short to implement, robust across sample sizes, and suitable for automated pipelines and exploratory analysis at scale because of its reliability and speed.
Paper Structure (14 sections, 1 theorem, 8 equations, 7 figures)

This paper contains 14 sections, 1 theorem, 8 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Related work
  3. Histograms, kernels, and mixtures.
  4. Laplacian smoothing and spectral viewpoints.
  5. Laplacians with potentials.
  6. Orthogonal projection based estimators
  7. The basic method
  8. Exploratory analysis
  9. Automated selection of the number of eigenvalues
  10. Testing on real data
  11. Summary, discussion, and future work
  12. Alternative basis construction
  13. Alternatives to projection
  14. Estimator code snippet

Key Result

Theorem 1

Let ${\bm{A}}$ be a real symmetric $n \times n$ matrix, let $\lambda_1 < \dots < \lambda_n$ be its eigenvalues, and let ${\bm{e}}_1, \dots, {\bm{e}}_n$ be the corresponding orthonormal eigenvectors. Then,

Figures (7)

  • Figure 1: Empirical frequency vectors (gray) and underlying PMFs that generated them (orange).
  • Figure 2: PMFs, samples, and eigenvectors. The left-most column has the true PMF supported on $0, 1, \dots, 999$ in orange, and empirical frequencies ${\bm{p}}$ obtained from 500 samples in blue. The next columns depict the first five eigenvectors of ${\bm{L}} - \mathop{\mathrm{diag}}\nolimits({\bm{p}})$.
  • Figure 3: The projections of the empirical frequencies stemming from 500 samples from $\{0, 1, \dots, 4999\}$ onto the first $k$ eigenvectors of ${\bm{L}} - \mathop{\mathrm{diag}}\nolimits({\bm{p}})$ for different values of $k$. Rows are various distribution shapes. The first column depicts true and empirical PMFs. The next columns are the projections of this empirical PMF onto eigenspaces of increasing dimension, depicted in blue.
  • Figure 4: The projections of the empirical histograms stemming from samples from $\{0, 1, \dots, 4999\}$ of various sizes onto the first $k=16$ eigenvectors of ${\bm{L}} - \mathop{\mathrm{diag}}\nolimits({\bm{p}})$. Rows are various distribution shapes. Columns represent sample sizes. Our method in blue, whereas the true density in yellow.
  • Figure 5: Fits obtained for different sample sizes with out method (blue), Gaussian kernel-density estimate (red), and logspline (green). The true PMF is in yellow. Rows are different distributions, whereas columns are different sample sizes.
  • ...and 2 more figures

Theorems & Definitions (1)

  • Theorem 1