Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Maximum Ideal Likelihood Estimation: A Unified Inference Framework for Latent Variable Models

Yizhou Cai, Ting Fung Ma

TL;DR

The paper introduces Maximum Ideal Likelihood Estimation (MILE), a unified framework for latent-variable models that parameterises latent variables and maximises the joint ideal likelihood $L(\bm{Z}, \bm{\theta}|\bm{X}) = f(\bm{X}, \bm{Z}|\bm{\theta})$, enabling simultaneous estimation of parameters and latent states. It establishes a rigorous inference foundation with conditional, joint, and marginal asymptotic normality under mild regularity conditions, and presents flexible computational strategies (Block Coordinate Ascend, Stepwise Categorical Progress, and Genetic Algorithms) for differentiable and non-differentiable, continuous and categorical latent variables. Through extensive simulations across Beta-Bernoulli, Log-Cauchy, Gaussian mixtures, and Bayesian segmented regression, MILE demonstrates competitive or superior statistical efficiency, faster computation, and scalable latent-variable recovery compared to EM-type methods and MCMC, including scenarios where traditional methods fail. The framework also shows compatibility with distributional approximations (VB, INLA, Vecchia) and practical applicability to real data, as illustrated by a Land Surface Temperature–ecoregion analysis. Overall, MILE provides a robust, flexible alternative to conventional latent-variable inference with strong theoretical guarantees and practical scalability.

Abstract

This paper develops a unified estimation framework, the Maximum Ideal Likelihood Estimation (MILE), for general parametric models with latent variables. Unlike traditional approaches relying on the marginal likelihood of the observed data, MILE directly exploits the joint distribution of the complete data by treating the latent variables as parameters (the ideal likelihood). Borrowing strength from optimisation techniques and algorithms, MILE is a broadly applicable framework in case that traditional methods fail, such as when the marginal likelihood has non-finite expectations. MILE offers a flexible and robust alternative to established techniques, including the Expectation-Maximisation algorithm and Markov chain Monte Carlo. We facilitate statistical inference of MILE on consistency, asymptotic distribution, and equivalence to the Maximum Likelihood Estimation, under some mild conditions. Extensive simulations illustrative real-data applications illustrate the empirical advantages of MILE, outperforming existing methods on computational feasibility and scalability.

Maximum Ideal Likelihood Estimation: A Unified Inference Framework for Latent Variable Models

TL;DR

The paper introduces Maximum Ideal Likelihood Estimation (MILE), a unified framework for latent-variable models that parameterises latent variables and maximises the joint ideal likelihood , enabling simultaneous estimation of parameters and latent states. It establishes a rigorous inference foundation with conditional, joint, and marginal asymptotic normality under mild regularity conditions, and presents flexible computational strategies (Block Coordinate Ascend, Stepwise Categorical Progress, and Genetic Algorithms) for differentiable and non-differentiable, continuous and categorical latent variables. Through extensive simulations across Beta-Bernoulli, Log-Cauchy, Gaussian mixtures, and Bayesian segmented regression, MILE demonstrates competitive or superior statistical efficiency, faster computation, and scalable latent-variable recovery compared to EM-type methods and MCMC, including scenarios where traditional methods fail. The framework also shows compatibility with distributional approximations (VB, INLA, Vecchia) and practical applicability to real data, as illustrated by a Land Surface Temperature–ecoregion analysis. Overall, MILE provides a robust, flexible alternative to conventional latent-variable inference with strong theoretical guarantees and practical scalability.

Abstract

This paper develops a unified estimation framework, the Maximum Ideal Likelihood Estimation (MILE), for general parametric models with latent variables. Unlike traditional approaches relying on the marginal likelihood of the observed data, MILE directly exploits the joint distribution of the complete data by treating the latent variables as parameters (the ideal likelihood). Borrowing strength from optimisation techniques and algorithms, MILE is a broadly applicable framework in case that traditional methods fail, such as when the marginal likelihood has non-finite expectations. MILE offers a flexible and robust alternative to established techniques, including the Expectation-Maximisation algorithm and Markov chain Monte Carlo. We facilitate statistical inference of MILE on consistency, asymptotic distribution, and equivalence to the Maximum Likelihood Estimation, under some mild conditions. Extensive simulations illustrative real-data applications illustrate the empirical advantages of MILE, outperforming existing methods on computational feasibility and scalability.
Paper Structure (27 sections, 5 theorems, 25 equations, 5 figures, 5 tables, 1 algorithm)

This paper contains 27 sections, 5 theorems, 25 equations, 5 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Traditional Approaches to Model with Latent Variables
  4. Motivations
  5. MILE Framework Set-up
  6. Comparisons between Common Methods
  7. Philosophical Ideas and Mathematical Nature
  8. Implementation and Methodology
  9. Proposed Methodology
  10. Computational Procedures
  11. Continuous Latent optimisation
  12. Categorical Optimisation
  13. Inference with Interpretation
  14. Assumptions
  15. Robust Parametric Normality
  16. ...and 12 more sections

Key Result

Theorem 1

SCP converges to $\widehat{\bm{Z}}$, the unique local maximiser of the fitness function $g(\bm{Z})$, if $g(\bm{Z})$ is slice-wise convex with categorical input and finite domain.

Figures (5)

  • Figure 1: Density Plot, Accuracy of MILE and the EM prediction
  • Figure 2: Histogram of the Bias of Change Point Estimates, $N$ = 10
  • Figure 3: $M$ = 20, $N$ = 10
  • Figure 4: $M$ = 200, $N$ = 10
  • Figure 5: Evolution of cluster assignments for U.S. LST Data under SCP. Each panel displays current grouping as latent variables. Panels are arranged from left to right, top to bottom, with 2000 iterations between adjacent panels.

Theorems & Definitions (9)

  • Definition 1
  • Definition 2
  • Definition 3
  • Theorem 1
  • proof
  • Theorem 2
  • Theorem 3
  • Corollary 1
  • Theorem 4