Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Two-stage Estimation of Latent Variable Regression Models: A General, Root-N Consistent Solution

Yang Liu, Xiaohui Luo, Jieyuan Dong, Youjin Sung, Yueqin Hu, Hongyun Liu, Daniel J. Bauer

TL;DR

This paper addresses the bias inherent in two-stage estimators for latent variable models, particularly factor score regression, by introducing a general bias-correction framework that preserves the computational advantages of two-stage approaches. It casts the problem in a two-stage parameterization with nuisance and focal components, and develops a black-box, stochastic-approximation procedure to produce bias-corrected focal estimates via an invertible mapping from the expected two-stage estimator. The authors establish $\,\sqrt{n}$-consistency under mild regularity, and provide practical point-estimation (Robbins-Monro) and variance-estimation (Monte Carlo-based ACM) algorithms, requiring only a data-generating algorithm and a $\sqrt{n}$-consistent nuisance estimator. Through three simulation studies of increasing LV-model complexity (simple latent regression, latent moderation, and multiple IRT-based LDVs), the bias-corrected FSR performs comparably to one-stage ML in both bias and variance, while avoiding the computational burden of full ML in many settings. The results suggest that bias-corrected two-stage estimation offers a robust, scalable alternative for estimating LV models with broader applicability than existing methods, with practical implications for psychological measurement and theory testing.

Abstract

Latent variable (LV) models are widely used in psychological research to investigate relationships among unobservable constructs. When one-stage estimation of the overall LV model is challenging, two-stage factor score regression (FSR) serves as a convenient alternative: the measurement model is fitted to obtain factor scores in the first stage, which are then used to fit the structural model in the subsequent stage. However, naive application of FSR is known to yield biased estimates of structural parameters. In this paper, we develop a generic bias-correction framework for two-stage estimation of parametric statistical models and tailor it specifically to FSR. Unlike existing bias-corrected FSR solutions, the proposed method applies to a broader class of LV models and does not require computing specific types of factor scores. We establish the root-n consistency of the proposed bias-corrected two-stage estimator under mild regularity conditions. To ensure broad applicability and minimize reliance on complex analytical derivations, we introduce a stochastic approximation algorithm for point estimation and a Monte Carlo-based procedure for variance estimation. In a sequence of Monte Carlo experiments, we demonstrate that the bias-corrected FSR estimator performs comparably to the ``gold standard'' one-stage maximum likelihood estimator. These results suggest that our approach offers a straightforward yet effective alternative for estimating LV models.

Two-stage Estimation of Latent Variable Regression Models: A General, Root-N Consistent Solution

TL;DR

This paper addresses the bias inherent in two-stage estimators for latent variable models, particularly factor score regression, by introducing a general bias-correction framework that preserves the computational advantages of two-stage approaches. It casts the problem in a two-stage parameterization with nuisance and focal components, and develops a black-box, stochastic-approximation procedure to produce bias-corrected focal estimates via an invertible mapping from the expected two-stage estimator. The authors establish -consistency under mild regularity, and provide practical point-estimation (Robbins-Monro) and variance-estimation (Monte Carlo-based ACM) algorithms, requiring only a data-generating algorithm and a -consistent nuisance estimator. Through three simulation studies of increasing LV-model complexity (simple latent regression, latent moderation, and multiple IRT-based LDVs), the bias-corrected FSR performs comparably to one-stage ML in both bias and variance, while avoiding the computational burden of full ML in many settings. The results suggest that bias-corrected two-stage estimation offers a robust, scalable alternative for estimating LV models with broader applicability than existing methods, with practical implications for psychological measurement and theory testing.

Abstract

Latent variable (LV) models are widely used in psychological research to investigate relationships among unobservable constructs. When one-stage estimation of the overall LV model is challenging, two-stage factor score regression (FSR) serves as a convenient alternative: the measurement model is fitted to obtain factor scores in the first stage, which are then used to fit the structural model in the subsequent stage. However, naive application of FSR is known to yield biased estimates of structural parameters. In this paper, we develop a generic bias-correction framework for two-stage estimation of parametric statistical models and tailor it specifically to FSR. Unlike existing bias-corrected FSR solutions, the proposed method applies to a broader class of LV models and does not require computing specific types of factor scores. We establish the root-n consistency of the proposed bias-corrected two-stage estimator under mild regularity conditions. To ensure broad applicability and minimize reliance on complex analytical derivations, we introduce a stochastic approximation algorithm for point estimation and a Monte Carlo-based procedure for variance estimation. In a sequence of Monte Carlo experiments, we demonstrate that the bias-corrected FSR estimator performs comparably to the ``gold standard'' one-stage maximum likelihood estimator. These results suggest that our approach offers a straightforward yet effective alternative for estimating LV models.
Paper Structure (23 sections, 1 theorem, 16 equations, 2 figures, 2 tables, 2 algorithms)

This paper contains 23 sections, 1 theorem, 16 equations, 2 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Statistical Framework of Two-Stage Estimation
  3. Two-Stage Estimation of Statistical Models
  4. Factor Score Regression
  5. Existing Bias-Corrected Two-Stage Estimators for Latent Variable Models
  6. A General Bias-Correction Strategy
  7. Algorithms
  8. Point Estimation
  9. Asymptotic Covariance Matrix
  10. Simulation Studies
  11. Study 1: Simple Latent Regression with Continuous Manifest Variables
  12. Data Generation
  13. Candidate Estimators
  14. Results
  15. Study 2: Latent Moderation Analysis with Continuous Manifest Variables
  16. ...and 8 more sections

Key Result

Proposition 1

Suppose that the nuisance parameter estimator $\hat{\boldsymbol{\nu}}(\mathbf{Y}_{1:n})$ and the initial focal parameter estimator $\hat{\boldsymbol{\varphi}}(\mathbf{Y}_{1:n}; \hat{\boldsymbol{\nu}}(\mathbf{Y}_{1:n}))$ satisfy under true model with parameters $\boldsymbol{\theta}^* = (\boldsymbol{\nu}^*{}^\top, \boldsymbol{\varphi}^*{}^\top)^\top$, in which $\mathbf{0}_q$ is a $q\times 1$ vector

Figures (2)

  • Figure 1: Graphical illustration of the bias-correction strategy in a single-parameter model. The parameter is denoted by $\varphi$. Panel A: The expected value of the initial (possibly biased) estimator $\hat{\varphi}(\mathbf{Y}_{1:n})$, $h(\varphi) = \mathbb{E}_{\varphi}\hat{\varphi}(\mathbf{Y}_{1:n})$, is plotted as a monotonic function of $\varphi$ (shown as the thick curve). Given the observed data $\mathbf{y}_{1:n}$, the bias-corrected estimate, denoted $\hat{\varphi}_{\rm BC}(\mathbf{y}_{1:n})$, is obtained by mapping the initial estimate $\hat{\varphi}(\mathbf{y}_{1:n})$ from the $y$-axis back to the $x$-axis using the curve $h$. The corresponding sampling distribution of the bias-corrected estimator (i.e., the bell-shaped curve on the $x$-axis) is induced by the sampling distribution of the initial estimator (i.e., the bell-shaped curve on the $y$-axis). Panel B: The bias-corrected estimator is more variable if the initial estimator is more variable, holding the rate of change of $h$ constant at the data generating $\varphi^*$. Panel C: The bias-corrected estimator is more variable if the function $h$ is flatter at $\varphi^*$, holding the variability of the initial estimator constant.
  • Figure 2: Path diagrams of data generating models. Latent and manifest variables are respectively represented by circles and rectangles. Regression paths and covariances are respectively depicted as single- and double-sided arrows. Nuisance parameters, which are estimated in the first stage of factor score regression, are shown in gray, while focal parameters, which are estimated in the second stage, are shown in black. Panel A: Simple latent regression with continuous manifest variables. Panel B: Latent moderation analysis with continuous manifest variables. Panel C: Multiple latent regression with dichotomous manifest variables.

Theorems & Definitions (10)

  • Proposition 1
  • proof
  • Remark 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Remark 5
  • Remark 6
  • Remark 7
  • Remark 8