Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the inverse of covariance matrices for unbalanced crossed designs

Ziyang Lyu, S. A. Sisson, A. H. Welsh

TL;DR

This work tackles the long-standing problem of obtaining an analytic inverse for the marginal covariance $\mathbf{V}$ in linear mixed models with crossed random effects under unbalanced designs. By representing the covariance structure with the Khatri--Rao product and introducing a modified covariance $\check{\mathbf{V}}$, the authors derive an exact spectral decomposition and a tractable closed-form inverse, enabling both asymptotic approximations and non-asymptotic refinements. They show that a simple block-diagonal approximation suffices in mildly unbalanced regimes, while a Sherman--Morrison-type recursion provides an exact inverse via rank-one updates for severely unbalanced cases. The approach yields accurate, scalable likelihood-based inference and paves the way for extensions to higher-order crossed designs, with clear practical impact for ML/REML estimation and EBLUP in complex, real-world data.

Abstract

This paper addresses a long-standing open problem in the analysis of linear mixed models with crossed random effects under unbalanced designs: how to find an analytic expression for the inverse of $\mathbf{V}$, the covariance matrix of the observed response. The inverse matrix $\mathbf{V}^{-1}$ is required for likelihood-based estimation and inference. However, for unbalanced crossed designs, $\mathbf{V}$ is dense and the lack of a closed-form representation for $\mathbf{V}^{-1}$, until now, has made using likelihood-based methods computationally challenging and difficult to analyse mathematically. We use the Khatri--Rao product to represent $\mathbf{V}$ and then to construct a modified covariance matrix whose inverse admits an exact spectral decomposition. Building on this construction, we obtain an elegant and simple approximation to $\mathbf{V}^{-1}$ for asymptotic unbalanced designs. For non-asymptotic settings, we derive an accurate and interpretable approximation under mildly unbalanced data and establish an exact inverse representation as a low-rank correction to this approximation, applicable to arbitrary degrees of unbalance. Simulation studies demonstrate the accuracy, stability, and computational tractability of the proposed framework.

On the inverse of covariance matrices for unbalanced crossed designs

TL;DR

This work tackles the long-standing problem of obtaining an analytic inverse for the marginal covariance in linear mixed models with crossed random effects under unbalanced designs. By representing the covariance structure with the Khatri--Rao product and introducing a modified covariance , the authors derive an exact spectral decomposition and a tractable closed-form inverse, enabling both asymptotic approximations and non-asymptotic refinements. They show that a simple block-diagonal approximation suffices in mildly unbalanced regimes, while a Sherman--Morrison-type recursion provides an exact inverse via rank-one updates for severely unbalanced cases. The approach yields accurate, scalable likelihood-based inference and paves the way for extensions to higher-order crossed designs, with clear practical impact for ML/REML estimation and EBLUP in complex, real-world data.

Abstract

This paper addresses a long-standing open problem in the analysis of linear mixed models with crossed random effects under unbalanced designs: how to find an analytic expression for the inverse of , the covariance matrix of the observed response. The inverse matrix is required for likelihood-based estimation and inference. However, for unbalanced crossed designs, is dense and the lack of a closed-form representation for , until now, has made using likelihood-based methods computationally challenging and difficult to analyse mathematically. We use the Khatri--Rao product to represent and then to construct a modified covariance matrix whose inverse admits an exact spectral decomposition. Building on this construction, we obtain an elegant and simple approximation to for asymptotic unbalanced designs. For non-asymptotic settings, we derive an accurate and interpretable approximation under mildly unbalanced data and establish an exact inverse representation as a low-rank correction to this approximation, applicable to arbitrary degrees of unbalance. Simulation studies demonstrate the accuracy, stability, and computational tractability of the proposed framework.

Paper Structure

This paper contains 22 sections, 16 theorems, 183 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Khatri--Rao product
  4. Notation
  5. Matrix form
  6. Inverse Covariance Matrix
  7. Spectral decomposition of a modified covariance matrix
  8. Inverse of covariance matrix $\mathbf{V}$
  9. Numerical Results
  10. Discussion
  11. Proofs
  12. Proof of Theorem \ref{['theom1']}
  13. Proof of Corollary \ref{['corollary1']}
  14. Proof of Theorem \ref{['theorem4']}
  15. Proof of Theorem \ref{['theorem2']}
  16. ...and 7 more sections

Key Result

Lemma 1

Suppose $\mathbf{a}$ is a $g \times 1$ vector and $\mathbf{b}$ is an $h \times 1$ vector. Let $\mathbf{P}$ and $\mathbf{R}$ be $g \times g$ matrices, and $\mathbf{Q}$ and $\mathbf{S}$ be $h \times h$ matrices. Then the following equalities hold. where $\mathbf{I}_{\bm}^f=\operatorname{diag}(f(m_{11})\mathbf{I}_{m_{11}},\ldots,f(m_{gh})\mathbf{I}_{m_{gh}})$, $f(\bm)=\operatorname{diag}(f(m_{11}),\

Figures (2)

  • Figure 1: Average Inversion Residual (AIR) for both asymptotic and non-asymptotic settings, with shaded areas representing one standard deviation across $N=200$ replications.
  • Figure 2: Non-asymptotic Setting: Average Inversion Residual with shaded areas representing one standard deviation

Theorems & Definitions (26)

  • Lemma 1
  • Theorem 1
  • Corollary 2
  • Theorem 3
  • Theorem 4
  • Theorem 5
  • proof
  • proof
  • proof
  • proof
  • ...and 16 more