Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Estimation of differential entropy for normal populations under prior information

Somnath Mandal, Lakshmi Kanta Patra

Abstract

The problem of nonlinear functional of parameters, such as differential entropy, has received much attention in information theory and statistics. In many situations, prior information about the parameters is available in the form of order restrictions. This information should be taken into account to obtain improved estimators. In this paper, we study the problems of point-wise and interval estimation of the entropy of two normal populations under a general location-invariant loss function. For the point-wise estimation, we have derived the maximum likelihood estimator (MLE), restricted MLE and the uniformly minimum variance unbiased estimator (UMVUE). Further, we derive a sufficient condition for improvement over affine equivariant estimators. A class of improved estimators is derived that dominates the best affine equivariant estimator (BAEE). Furthermore, we obtain a class of smooth improved estimator that dominates BAEE. We present special loss functions and derive expressions for the proposed improved estimators. A numerical study is conducted to compare the risk performance of the proposed estimators under quadratic and linex loss functions. For interval estimation, we have derived asymptotic confidence interval, bootstrap confidence intervals, HPD credible interval, and intervals based on generalized pivot variables. A comprehensive numerical comparison of these intervals is carried out in terms of coverage probabilities and average lengths. Finally, the proposed results are illustrated with a real example: the failure of the air-conditioning systems on Boeing 720 jet planes.

Estimation of differential entropy for normal populations under prior information

Abstract

The problem of nonlinear functional of parameters, such as differential entropy, has received much attention in information theory and statistics. In many situations, prior information about the parameters is available in the form of order restrictions. This information should be taken into account to obtain improved estimators. In this paper, we study the problems of point-wise and interval estimation of the entropy of two normal populations under a general location-invariant loss function. For the point-wise estimation, we have derived the maximum likelihood estimator (MLE), restricted MLE and the uniformly minimum variance unbiased estimator (UMVUE). Further, we derive a sufficient condition for improvement over affine equivariant estimators. A class of improved estimators is derived that dominates the best affine equivariant estimator (BAEE). Furthermore, we obtain a class of smooth improved estimator that dominates BAEE. We present special loss functions and derive expressions for the proposed improved estimators. A numerical study is conducted to compare the risk performance of the proposed estimators under quadratic and linex loss functions. For interval estimation, we have derived asymptotic confidence interval, bootstrap confidence intervals, HPD credible interval, and intervals based on generalized pivot variables. A comprehensive numerical comparison of these intervals is carried out in terms of coverage probabilities and average lengths. Finally, the proposed results are illustrated with a real example: the failure of the air-conditioning systems on Boeing 720 jet planes.
Paper Structure (19 sections, 18 theorems, 73 equations, 9 figures, 3 tables)

This paper contains 19 sections, 18 theorems, 73 equations, 9 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Improved estimation of $\tau$
  3. Restricted MLE
  4. Smooth improved estimator
  5. A class of improved estimators
  6. Improved estimation under Generalized Pitman Closeness
  7. Improve estimation of $\ln \sigma$
  8. A simulation study: comparing the risk performance of improved estimators
  9. Confidence interval estimation
  10. Asymptotic confidence intervals
  11. Parametric bootstrap confidence intervals
  12. Bootstrap-p method
  13. Bootstrap-t method
  14. Generalized confidence intervals
  15. HPD credible interval using MCMC method
  16. ...and 4 more sections

Key Result

Lemma 1.1

Under a location-invariant loss function $L(t)$, the best affine equivariant estimator (BAEE) of $\tau$ is given by: where $d_0$ is the unique solution of

Figures (9)

  • Figure 1: RRI of various estimators of $\ln\sigma$ with respect to BAEE under $L_1(t)$
  • Figure 2: RRI of various estimators $\ln\sigma$ with respect to BAEE under $L_2(t)$ for $a_1=-3$
  • Figure 3: RRI of RMLE w.r.t MLE for sample sizes $n=5,8,12,$ and $18$ under $L_1(t)$
  • Figure 4: RRI of RMLE w.r.t MLE for sample sizes $n=5,8,12,$ and $18$ under $L_2(t)$ for various values of $a_1$
  • Figure 5: Asymptotic confidence interval
  • ...and 4 more figures

Theorems & Definitions (30)

  • Lemma 1.1
  • Remark 1.1
  • Example 1.1
  • Example 1.2
  • Remark 1.2
  • Theorem 1.2
  • Theorem 1.3
  • Theorem 2.1
  • Theorem 2.2
  • Example 2.1
  • ...and 20 more