Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

New closed-form estimators for discrete distributions

Adrian Fischer

TL;DR

This work tackles parameter estimation for discrete distributions on $\mathbb{Z}^d$ when normalizing constants are intractable. It adopts a discrete density approach to Stein's Method of Moments, yielding estimators that often have closed-form expressions via a Stein operator $\mathcal{A}_{\theta}$ and a test-function class $\mathscr{F}$. The authors develop univariate and multivariate Stein estimators, demonstrate strong performance in small samples, and extend the framework to truncated domains and even unknown truncation boundaries with theoretical guarantees on consistency and asymptotic normality. Through extensive simulations, the Stein estimators show competitive bias and MSE relative to MLE while offering computational simplicity and robustness, especially in high dimensions or when numerical optimization is difficult. Overall, the paper provides a practical, normalization-constant-free estimation toolkit for a broad class of discrete models, including truncated and multivariate cases.

Abstract

We revisit the problem of parameter estimation for discrete probability distributions with values in $\mathbb{Z}^d$. To this end, we adapt a technique called Stein's Method of Moments to discrete distributions which often gives closed-form estimators when standard methods such as maximum likelihood estimation (MLE) require numerical optimization. These new estimators exhibit good performance in small-sample settings which is demonstrated by means of a comparison to the MLE through simulation studies. We pay special attention to truncated distributions and show that the asymptotic behavior of our estimators is not affected by an unknown (rectangular) truncation domain.

New closed-form estimators for discrete distributions

TL;DR

This work tackles parameter estimation for discrete distributions on when normalizing constants are intractable. It adopts a discrete density approach to Stein's Method of Moments, yielding estimators that often have closed-form expressions via a Stein operator and a test-function class . The authors develop univariate and multivariate Stein estimators, demonstrate strong performance in small samples, and extend the framework to truncated domains and even unknown truncation boundaries with theoretical guarantees on consistency and asymptotic normality. Through extensive simulations, the Stein estimators show competitive bias and MSE relative to MLE while offering computational simplicity and robustness, especially in high dimensions or when numerical optimization is difficult. Overall, the paper provides a practical, normalization-constant-free estimation toolkit for a broad class of discrete models, including truncated and multivariate cases.

Abstract

We revisit the problem of parameter estimation for discrete probability distributions with values in . To this end, we adapt a technique called Stein's Method of Moments to discrete distributions which often gives closed-form estimators when standard methods such as maximum likelihood estimation (MLE) require numerical optimization. These new estimators exhibit good performance in small-sample settings which is demonstrated by means of a comparison to the MLE through simulation studies. We pay special attention to truncated distributions and show that the asymptotic behavior of our estimators is not affected by an unknown (rectangular) truncation domain.
Paper Structure (4 sections, 5 theorems, 72 equations, 1 figure, 9 tables)

This paper contains 4 sections, 5 theorems, 72 equations, 1 figure, 9 tables.

Table of Contents

  1. Introduction
  2. Estimation
  3. Multivariate distributions
  4. Unknown truncation domain

Key Result

Theorem 2.1

Let $X \sim \mathbb{P}_{\theta}$. Then, for all $f \in \mathscr{F}$.

Figures (1)

  • Figure 1: Relative asymptotic efficiency of the Stein estimator $V^{\mathrm{ST}}(p)/V^{\mathrm{ML}}(p)$ as a function of $p$ for the logarithmic distribution.

Theorems & Definitions (21)

  • Theorem 2.1
  • Remark 2.2
  • Remark 2.3
  • Remark 2.4
  • Example 2.5
  • Example 2.6
  • Example 2.7
  • Example 2.8
  • Example 2.9
  • Example 2.10
  • ...and 11 more