Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Statistical Inference for Random Unknowns via Modifications of Extended Likelihood

Hangbin Lee, Youngjo Lee

TL;DR

This paper extends Fisher's likelihood framework to models with both fixed unknowns and random unknowns by introducing a modified extended likelihood, the h-likelihood, and a corresponding h-confidence. Maximizing the h-likelihood yields maximum h-likelihood estimators (MHLEs) for fixed parameters and best unbiased predictors for random effects, achieving a generalized Cramér-Rao bound ($GCRLB$) and enabling scalable, accurate inference in large and complex models. The h-confidence extends the confidence distribution concept to accommodate random unknowns, producing exact or near-exact interval predictions for both fixed and random components, even in small samples. Approximation methods (e.g., Enhanced Laplace Approximation and C1–C3 constructions) broaden applicability to broad model classes while preserving asymptotic validity of inference, positioning the approach as a practical alternative to marginal and traditional extended-likelihood methods for modern large-scale problems with unobservables.

Abstract

Fisher's likelihood is widely used for statistical inference for fixed unknowns. This paper aims to extend two important likelihood-based methods, namely the maximum likelihood procedure for point estimation and the confidence procedure for interval estimation, to embrace a broader class of statistical models with additional random unknowns. We propose the new h-likelihood and the h-confidence by modifying extended likelihoods. Maximization of the h-likelihood yields both maximum likelihood estimators of fixed unknowns and asymptotically optimal predictors for random unknowns, achieving the generalized Cramér-Rao lower bound. The h-likelihood further offers advantages in scalability for large datasets and complex models. The h-confidence could yield a valid interval estimation and prediction by maintaining the coverage probability for both fixed and random unknowns in small samples. We study approximate methods for the h-likelihood and h-confidence, which can be applied to a general class of models with additional random unknowns.

Statistical Inference for Random Unknowns via Modifications of Extended Likelihood

TL;DR

This paper extends Fisher's likelihood framework to models with both fixed unknowns and random unknowns by introducing a modified extended likelihood, the h-likelihood, and a corresponding h-confidence. Maximizing the h-likelihood yields maximum h-likelihood estimators (MHLEs) for fixed parameters and best unbiased predictors for random effects, achieving a generalized Cramér-Rao bound () and enabling scalable, accurate inference in large and complex models. The h-confidence extends the confidence distribution concept to accommodate random unknowns, producing exact or near-exact interval predictions for both fixed and random components, even in small samples. Approximation methods (e.g., Enhanced Laplace Approximation and C1–C3 constructions) broaden applicability to broad model classes while preserving asymptotic validity of inference, positioning the approach as a practical alternative to marginal and traditional extended-likelihood methods for modern large-scale problems with unobservables.

Abstract

Fisher's likelihood is widely used for statistical inference for fixed unknowns. This paper aims to extend two important likelihood-based methods, namely the maximum likelihood procedure for point estimation and the confidence procedure for interval estimation, to embrace a broader class of statistical models with additional random unknowns. We propose the new h-likelihood and the h-confidence by modifying extended likelihoods. Maximization of the h-likelihood yields both maximum likelihood estimators of fixed unknowns and asymptotically optimal predictors for random unknowns, achieving the generalized Cramér-Rao lower bound. The h-likelihood further offers advantages in scalability for large datasets and complex models. The h-confidence could yield a valid interval estimation and prediction by maintaining the coverage probability for both fixed and random unknowns in small samples. We study approximate methods for the h-likelihood and h-confidence, which can be applied to a general class of models with additional random unknowns.
Paper Structure (46 sections, 9 theorems, 214 equations, 8 figures)

This paper contains 46 sections, 9 theorems, 214 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Backgrounds
  3. Marginal likelihood approach
  4. Extended likelihood approaches
  5. Motivation of modified extended likelihoods
  6. New h-likelihood
  7. Bartlizable scale of random parameters
  8. Geometric Insights into h-Likelihood under LP and ELP
  9. Numerical studies for scalability
  10. Asymptotic properties of MHLEs
  11. Generalized Cramér-Rao lower bound
  12. Asymptotic normality of MHLEs
  13. Numerical studies for convergence
  14. Small sample properties of MHLEs
  15. Missing data problems with $m=0$
  16. ...and 31 more sections

Key Result

Lemma 1

(a) For any continuous random parameters $(u_{1},...,u_{n})^{\intercal}$, there exists a one-to-one transformation to the Bartlizable scale $v_{i}=g_{v}(u_{i})$. (b) The scale $\mathbf{v}=(v_{1},...,v_{n})^{\intercal}$ is Bartlizable if (c) The scale $\mathbf{v}$ is Bartlizable if $f_{\boldsymbol{\theta}}(\mathbf{v})$ is differentiable for any $\mathbf{v}\in \Omega_{\mathbf{v}}=\mathbb{R}^{n}$.

Figures (8)

  • Figure 1: The h-likelihood (blue), the marginal likelihood (red), and the profile predictive likelihood (orange) when $y_{i}\sim N(x_{i} \theta +v,1)$ and $v\sim N(0,1)$ for $i=1,..., 2000$.
  • Figure 2: RMSE curves of marginal likelihood (gray) and h-likelihood (black) approaches from 10 repetitions.
  • Figure 3: Box-plots for the estimation errors of fixed and random unknowns in LMM.
  • Figure 4: Coverage probabilities of 90% CI for $\theta$ in Example 3 where the true value of $\theta$ varies from 0.01 to 5.
  • Figure 6: Marginal coverage probabilities of 90% PI for future $u$ in Example 3, based on pivotal quantities (solid), plug-in method (dotted) and Wald PI (dashed), where the true value of $\theta$ varies.
  • ...and 3 more figures

Theorems & Definitions (11)

  • Lemma 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Theorem 5
  • Theorem 6
  • Corollary 7
  • Theorem 8
  • proof
  • Lemma 9
  • ...and 1 more