Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Local moment matching with Erlang mixtures under automatic roughness penalization

Oskar Laverny, Philippe Lambert

TL;DR

It is shown that the corresponding hyperparameter can be selected without cross-validation through the computation of the so-called effective dimension of the estimator, which makes the estimator practical and adapted to these summarized information settings.

Abstract

We consider the class of Erlang mixtures for the task of density estimation on the positive real line when the only available information is given as local moments, a histogram with potentially higher order moments in some bins. By construction, the obtained moment problem is ill-posed and requires regularization. Several penalties can be used for such a task, such as a lasso penalty for sparsity of the representation, but we focus here on a simplified roughness penalty from the P-splines literature. We show that the corresponding hyperparameter can be selected without cross-validation through the computation of the so-called effective dimension of the estimator, which makes the estimator practical and adapted to these summarized information settings. The flexibility of the local moments representations allows interesting additions such as the enforcement of Value-at-Risk and Tail Value-at-Risk constraints on the resulting estimator, making the procedure suitable for the estimation of heavy-tailed densities.

Local moment matching with Erlang mixtures under automatic roughness penalization

TL;DR

It is shown that the corresponding hyperparameter can be selected without cross-validation through the computation of the so-called effective dimension of the estimator, which makes the estimator practical and adapted to these summarized information settings.

Abstract

We consider the class of Erlang mixtures for the task of density estimation on the positive real line when the only available information is given as local moments, a histogram with potentially higher order moments in some bins. By construction, the obtained moment problem is ill-posed and requires regularization. Several penalties can be used for such a task, such as a lasso penalty for sparsity of the representation, but we focus here on a simplified roughness penalty from the P-splines literature. We show that the corresponding hyperparameter can be selected without cross-validation through the computation of the so-called effective dimension of the estimator, which makes the estimator practical and adapted to these summarized information settings. The flexibility of the local moments representations allows interesting additions such as the enforcement of Value-at-Risk and Tail Value-at-Risk constraints on the resulting estimator, making the procedure suitable for the estimation of heavy-tailed densities.
Paper Structure (11 sections, 22 equations, 6 figures, 3 tables, 1 algorithm)

This paper contains 11 sections, 22 equations, 6 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Proposed estimation scheme
  3. Numerical examples
  4. Lognormal example
  5. A mixture of gamma distributions
  6. A more challenging example
  7. A mixture of a (truncated) normal and a beta random variables.
  8. Discussion
  9. Acknowledgements
  10. Commonly used probability distributions
  11. Key evaluation statistics

Figures (6)

  • Figure 1: General results on \ref{['ex:lognormal']}. The histogram in (a) represents $\hat{\boldsymbol \pi}$, while the red curve represents the density estimate and the red region the (pointwise) confidence interval around the estimated density. Couples $(x_i,\omega_iy_i)$ from Equation \ref{['eq:couples']} are represented in panel (b). Panel (c) represents quantile-quantile plots of the estimated density against the underlying raw data and the true density. Panels (d) represents the median of the distance statistics across resamples for an increasing number of observed local moments, renormalized to be $1$ for the least informative setting.
  • Figure 2: Comparison of the bootstrap statistics obtained on \ref{['ex:lognormal']}. Panel (a) : for an increasing number of moments with $\bm k \in \{(1,1,1,1),(2,2,2,1),(3,3,3,1),(4,4,4,1)\}$ and a fixed number of observations $N$. Panel (b): for an increasing number of observations $N \in \{250,500,750,1000,2000\}$ and fixed $\bm k=(4,4,4,1)$.
  • Figure 3: General results on \ref{['ex:MixGamma1']}. The histogram in (a) represents $\hat{\boldsymbol \pi}$, while the red curve represents the density estimate and the red region the (pointwise) confidence interval around the estimated density. Couples $(x_i,\omega_iy_i)$ from Equation \ref{['eq:couples']} are represented in panel (b). Panel (c) represents quantile-quantile plots of the estimated density against the underlying raw data and the true density.
  • Figure 4: General results on \ref{['ex:GaussRevGamma500']}. The histogram in (a) represents $\hat{\boldsymbol \pi}$, while the red curve represents the density estimate and the red region the (pointwise) confidence interval around the estimated density. Couples $(x_i,\omega_iy_i)$ from Equation \ref{['eq:couples']} are represented in panel (b). Panel (c) represents quantile-quantile plots of the estimated density against the underlying raw data and the true density. Panels (d) represents the median of the distance statistics across resamples for an increasing number of observed local moments, renormalized to be $1$ for the least informative setting.
  • Figure 5: Comparison of the bootstrap statistics obtained on \ref{['ex:GaussRevGamma500']}. Panel (a) : for several parameters $\bm k \in \{(1,1,1,1),(2,2,2,1),(3,3,3,1),(4,4,4,1)\}$ and fixed number of observations $N$. Panel (b): for several number of observations $N \in \{250,500,1000,2000\}$ and fixed $\bm k=(4,4,4,1)$.
  • ...and 1 more figures