On Generalized Likelihood Estimation Based on the Logarithmic Norm Relative Entropy
Himanshi Singh, Abhik Ghosh, Nil Kamal Hazra
TL;DR
This work develops a robust parameter estimation framework based on the logarithmic norm relative entropy (LNRE), introducing the $\mathcal{M}^{(\alpha,\beta)}$-family which yields a fixed number of sufficient statistics under the generalized LNRE likelihood. It establishes a generalized Fisher-Darmois-Koopman-Pitman theorem, a generalized Rao-Blackwell theorem, and a generalized Cramér-Rao bound, and shows that the minimal sufficient statistic does not generally attain the bound. The authors derive explicit MLNREEs for the family of Student's distributions (both $t$ with $\nu>1$ and $r$ with $\nu<0$), including closed-form estimators for means and variances that remain robust under contamination. Through simulations and a real-data analysis (Newcomb's light speed data), they demonstrate improved robustness of MLNREE over traditional minimum DPD/LDPD estimators, with practical guidance on tuning via the parameter $\beta$. The work also corrects an error in a related Generalized sufficiency result and highlights the potential for further theoretical and applied development in robust, divergence-based inference.
Abstract
Traditional likelihood based methods for parameter estimation get highly affected when the given data is contaminated by outliers even in a small proportion. In this paper, we consider a robust parameter estimation method, namely the minimum logarithmic norm relative entropy (LNRE) estimation procedure, and study different (generalized) sufficiency principles associated with it. We introduce a new two-parameter power-law family of distributions (namely, $\mathcal{M}^{(α,β)}$-family), which is shown to have a fixed number of sufficient statistics, independent of the sample size, with respect to the generalized likelihood function associated with the LNRE. Then, we obtain the generalized minimal sufficient statistic for this family and derive the generalized Rao-Blackwell theorem and the generalized Cramér-Rao lower bound for the minimum LNRE estimation. We also study the minimum LNRE estimators (MLNREEs) for the family of Student's distributions particularly in detail. Our general results reduces to the classical likelihood based results under the exponential family of distributions at specific choices of the tuning parameter $α$ and $β$. Finally, we present simulation studies followed by a real data analysis, which highlight the practical utility of the MLNREEs for data contaminated by possible outliers. Along the way we also correct a mistake found in a recent paper on related theory of generalized likelihoods.