Bias-Reduced Estimation of Finite Mixtures: An Application to Latent Group Structures in Panel Data

TL;DR

The paper addresses substantial finite-sample bias in maximum likelihood estimation of finite mixture models, showing that tail-driven outliers and component overlap can misplace the global likelihood maximum. It introduces a bias-reduced estimation framework that maximizes the classification-mixture likelihood $l^{CM}$ using a consistent classifier, achieving improved finite-sample properties and potential oracle efficiency. Theoretical results establish conditions under which consistent estimation and asymptotic normality hold for the C-EM algorithm, complemented by Monte Carlo simulations across normal, Poisson, and exponential mixtures and a latent-group panel structure. An empirical application to health-care expenditures using a latent-group two-part model demonstrates notable out-of-sample improvements (17.6% relative to the best EM result) and recoverable group memberships, underscoring the method's practical relevance for policy-relevant heterogeneity analysis.

Abstract

Finite mixture models are widely used in econometric analyses to capture unobserved heterogeneity. This paper shows that maximum likelihood estimation of finite mixtures of parametric densities can suffer from substantial finite-sample bias in all parameters under mild regularity conditions. The bias arises from the influence of outliers in component densities with unbounded or large support and increases with the degree of overlap among mixture components. I show that maximizing the classification-mixture likelihood function, equipped with a consistent classifier, yields parameter estimates that are less biased than those obtained by standard maximum likelihood estimation (MLE). I then derive the asymptotic distribution of the resulting estimator and provide conditions under which oracle efficiency is achieved. Monte Carlo simulations show that conventional mixture MLE exhibits pronounced finite-sample bias, which diminishes as the sample size or the statistical distance between component densities tends to infinity. The simulations further show that the proposed estimation strategy generally outperforms standard MLE in finite samples in terms of both bias and mean squared errors under relatively weak assumptions. An empirical application to latent group panel structures using health administrative data shows that the proposed approach reduces out-of-sample prediction error by approximately 17.6% relative to the best results obtained from standard MLE procedures.

Figures (24)

• Figure 1: Hypothetical representation of the mixture likelihood $l^{ML}(\xi)$ when $N$ is small (Panel (a)) and $N$ is sufficiently large (Panel (b)).
• Figure 2: Simulation results for the mixture of normal distributions with true values $\boldsymbol{\mu}^0 = (0.25,-0.25)$, $\sigma^0 = (1,1)$, and $\pi^0_1 = 0.5$. The blue dots correspond to the estimated value averaged across 1,000 replications, while the error bars represent the $2.5^{th}$ and $97.5^{th}$ percentiles of the estimated parameter's empirical distribution. The red dashed lines correspond to the true parameter values.
• Figure 3: Simulation results for the mixture of normal distributions with true values $\boldsymbol{\mu}^0 = (0.75,-0.75)$, $\sigma^0 = (1,1)$, and $\pi^0_1 = 0.5$. The blue dots correspond to the estimated value averaged across 1,000 replications, while the error bars represent the $2.5^{th}$ and $97.5^{th}$ percentiles of the estimated parameter's empirical distribution. The red dashed lines correspond to the true parameter values.
• Figure 4: Estimation bias for all parameters in the mixture when $N = 500$, $T=5$, $G=2$, and $p=1$. The dots correspond to the estimated bias over 250 replications, while the error bars represent the $2.5^{th}$ and $97.5^{th}$ percentiles of the empirical distribution of the bias. The coefficients $Time1,....,Time5$ represent the time-fixed effects, with $Sigma2\_a = \sigma^2_{\alpha,g}$ and $Sigma2\_e = \sigma^2_{\epsilon}$.
• Figure 5: Estimation bias for all parameters in the mixture when $N = 500$, $T=5$, $G=2$, and $p=5$. The dots correspond to the estimated bias over 250 replications, while the error bars represent the $2.5^{th}$ and $97.5^{th}$ percentiles of the empirical distribution of the bias. The coefficients $Time1,....,Time5$ represent the time-fixed effects, with $Sigma2\_a = \sigma^2_{\alpha,g}$ and $Sigma2\_e = \sigma^2_{\epsilon}$.

Theorems & Definitions (33)

• Definition 1
• Example 1: Mixture of normal distributions
• Definition 2
• Definition 3
• Definition 4
• Definition 5
• Definition 6
• Lemma 1
• Lemma 2
• Theorem 1