Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Note on Likelihood Ratio Tests for Models with Latent Variables

Yunxiao Chen, Irini Moustaki, Haoran Zhang

TL;DR

This paper shows that Wilks' theorem can fail for likelihood ratio tests in latent-variable models, with irregular asymptotics arising from boundary and identifiability issues. It first illustrates the phenomenon via several latent-variable examples, then presents a unified Chernoff-regularity framework that yields correct nonstandard limits determined by tangent cones. The core results (Theorems LRT and LRT2) characterize the LRT's asymptotic distribution as either a chi-square or a mixture of chi-square distributions, depending on the local geometry of the parameter space. The work also discusses practical alternatives, such as parametric bootstrap or split LRT, and highlights the broad relevance to factor analysis, IFA, SEM, and random-effects settings. Overall, it provides a rigorous roadmap for valid inference when standard Wilks-based LRTs fail in latent-variable contexts.

Abstract

The likelihood ratio test (LRT) is widely used for comparing the relative fit of nested latent variable models. Following Wilks' theorem, the LRT is conducted by comparing the LRT statistic with its asymptotic distribution under the restricted model, a $χ^2$-distribution with degrees of freedom equal to the difference in the number of free parameters between the two nested models under comparison. For models with latent variables such as factor analysis, structural equation models and random effects models, however, it is often found that the $χ^2$ approximation does not hold. In this note, we show how the regularity conditions of Wilks' theorem may be violated using three examples of models with latent variables. In addition, a more general theory for LRT is given that provides the correct asymptotic theory for these LRTs. This general theory was first established in Chernoff (1954) and discussed in both van der Vaart (2000) and Drton (2009), but it does not seem to have received enough attention. We illustrate this general theory with the three examples.

A Note on Likelihood Ratio Tests for Models with Latent Variables

TL;DR

This paper shows that Wilks' theorem can fail for likelihood ratio tests in latent-variable models, with irregular asymptotics arising from boundary and identifiability issues. It first illustrates the phenomenon via several latent-variable examples, then presents a unified Chernoff-regularity framework that yields correct nonstandard limits determined by tangent cones. The core results (Theorems LRT and LRT2) characterize the LRT's asymptotic distribution as either a chi-square or a mixture of chi-square distributions, depending on the local geometry of the parameter space. The work also discusses practical alternatives, such as parametric bootstrap or split LRT, and highlights the broad relevance to factor analysis, IFA, SEM, and random-effects settings. Overall, it provides a rigorous roadmap for valid inference when standard Wilks-based LRTs fail in latent-variable contexts.

Abstract

The likelihood ratio test (LRT) is widely used for comparing the relative fit of nested latent variable models. Following Wilks' theorem, the LRT is conducted by comparing the LRT statistic with its asymptotic distribution under the restricted model, a -distribution with degrees of freedom equal to the difference in the number of free parameters between the two nested models under comparison. For models with latent variables such as factor analysis, structural equation models and random effects models, however, it is often found that the approximation does not hold. In this note, we show how the regularity conditions of Wilks' theorem may be violated using three examples of models with latent variables. In addition, a more general theory for LRT is given that provides the correct asymptotic theory for these LRTs. This general theory was first established in Chernoff (1954) and discussed in both van der Vaart (2000) and Drton (2009), but it does not seem to have received enough attention. We illustrate this general theory with the three examples.

Paper Structure

This paper contains 12 sections, 3 theorems, 48 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Literature on Likelihood Ratio Test
  3. Our Contributions
  4. Examples
  5. 1(a).
  6. 1(b).
  7. 2(a).
  8. 2(b).
  9. General Theory for Likelihood Ratio Test
  10. Testing Submodel against Saturated Model
  11. Comparing Two Nested Submodels
  12. Discussion

Key Result

Lemma 1

(1) For the two-factor model given in Example 1(a), choose the parameter space to be If the true parameters satisfy $a^*_{j2}=0, ~j=2,...,J,$ then $I(\boldsymbol \theta^*)$ is non-invertible. (2) For the two-factor IFA model given in Example 2(a), choose the parameter space to be $\Theta = \mathbb R^{3J-1}.$ If the true parameters satisfy $a^*_{j2}=0, ~j=2,...,J,$ then $I(\boldsymb

Figures (5)

  • Figure 1:
  • Figure 2:
  • Figure 4:
  • Figure 5:
  • Figure 7: The black solid line shows the empirical CDF of the LRT statistic, based on 5000 independent simulations. The red dashed line shows the CDF of the $\chi^2$-distribution with one-degree of freedom as suggested by Wilks' theorem. The blue dotted line shows the CDF of the mixture of $\chi^2$-distribution suggested by Theorem \ref{['thm:LRT2']}.

Theorems & Definitions (14)

  • Example 1
  • Example 2
  • Example 3
  • Lemma 1
  • Definition 1
  • Theorem 1
  • Remark 1
  • Remark 2
  • Example 4
  • Example 5
  • ...and 4 more