Rates of Convergence of Maximum Smoothed Log-Likelihood Estimators for Semi-Parametric Multivariate Mixtures

TL;DR

This work addresses parameter estimation in semi-parametric multivariate finite mixtures where each component density factors into univariate terms. It develops a smoothed log-likelihood framework, realized via an MM algorithm with a kernel-based smoothing operator that exponentiates a kernel convolution on log-densities, and establishes consistency for both finite-dimensional mixing proportions and infinite-dimensional component densities. The authors derive convergence rates that couple sample size, smoothing bandwidth, and the finite- versus infinite-dimensional parameter spaces, showing, for example, that with $h\sim C n^{-1/5}$ the finite-dimensional estimates converge at near $n^{-2/5}$ rates while the infinite-dimensional parts scale as $O_{\mathbb{P}}(n^{-1/2}h^{-1/2}+h^2)$. The results extend to variables on the real line under additional tail conditions and are corroborated by simulations, providing rigorous justification for the practical smoothed-likelihood approach in semi-parametric mixture modeling.

Abstract

Theoretical guarantees are established for a standard estimator in a semi-parametric finite mixture model, where each component density is modeled as a product of univariate densities under a conditional independence assumption. The focus is on the estimator that maximizes a smoothed log-likelihood function, which can be efficiently computed using a majorization-minimization algorithm. This smoothed likelihood applies a nonlinear regularization operator defined as the exponential of a kernel convolution on the logarithm of each component density. Consistency of the estimators is demonstrated by leveraging classical M-estimation frameworks under mild regularity conditions. Subsequently, convergence rates for both finite- and infinite-dimensional parameters are derived by exploiting structural properties of the smoothed likelihood, the behavior of the iterative optimization algorithm, and a thorough study of the profile smoothed likelihood. This work provides the first rigorous theoretical guarantees for this estimation approach, bridging the gap between practical algorithms and statistical theory in semi-parametric mixture modeling.

Theorems & Definitions (30)

• Lemma 1
• Lemma 2
• Theorem 1
• Lemma 3
• Remark 1
• Lemma 4
• Lemma 5
• Theorem 2
• Lemma 6
• Remark 2