Motivating REML via Prediction-Error Covariances in EM Updates for Linear Mixed Models

TL;DR

The paper provides a computational perspective on REML by embedding it in an EM framework for linear mixed models, showing that ML and REML solve the same Henderson's equations for BLUE and BLUP, with their divergence confined to trace adjustments in the M-step. The E-step calculations are identical across ML and REML, while the M-step uses different prediction-error covariances: ML relies on conditional covariances of the random effects given data, whereas REML uses prediction-error covariances that account for uncertainty in estimating the fixed effects. This yields explicit, easily teachable variance-component updates and clarifies REML's intuition, aided by short R code that reproduces ML and REML fits and aligns with lme4 results. The work emphasizes how REML improves variance-component estimation by incorporating uncertainty from fixed effects, offers classroom-friendly derivations, and provides practical code for hands-on exploration and validation.

Abstract

We present a computational motivation for restricted maximum likelihood (REML) estimation in linear mixed models using an expectation--maximization (EM) algorithm. At each iteration, maximum likelihood (ML) and REML solve the same mixed-model equations for the best linear unbiased estimator (BLUE) of the fixed effects and the best linear unbiased predictor (BLUP) of the random effects. They differ only in the trace adjustments used in the variance-component updates: ML uses conditional covariances of the random effects given the data, whereas REML uses prediction-error covariances from Henderson's C-matrix, reflecting uncertainty from estimating the fixed effects. Short R code makes this switch explicit, exposes the key matrices for classroom inspection, and reproduces lme4 ML and REML fits.