Bayes linear estimator in the general linear model
Hirai Mukasa
TL;DR
This paper develops a comprehensive theory of Bayes linear estimators in the general linear model, deriving the Bayes linear form $\hat{\boldsymbol{\beta}}_{BL}(\boldsymbol{\Phi},\boldsymbol{K})=\boldsymbol{K}\boldsymbol{X}^T(\boldsymbol{\Phi}+\boldsymbol{X}\boldsymbol{K}\boldsymbol{X}^T)^{-1}\boldsymbol{y}$ and connecting it to classical estimators such as ridge and generalized least squares. It establishes central properties (linear sufficiency and linear completeness) and presents necessary and sufficient conditions under which two Bayes linear estimators coincide for all data, revealing that equality is governed by Rao-type covariance structures: $\boldsymbol{\Omega}=\boldsymbol{X}\boldsymbol{\Gamma}\boldsymbol{X}^T+\boldsymbol{Z}(\boldsymbol{Z}^T\boldsymbol{Z})^{-1}\boldsymbol{Z}^T$. The work further characterizes the set of outcomes $\boldsymbol{y}$ for which estimator equality holds and derives equivalent conditions for the residual sums of squares under Bayes linear estimation, showing that substantial simplifications arise in these equalities under Rao's covariance form. Together, these results illuminate when Bayes linear procedures yield risk-efficient, potentially minimax, estimators and provide practical guidance for exploiting estimator equivalences in partially known covariance settings.
Abstract
The Bayes linear estimator is derived by minimizing the Bayes risk with respect to the squared loss function. Non-unbiased estimators such as ordinary ridge, typical shrinkage, fractional rank, and restricted least squares estimators, as well as classical linear unbiased estimators such as ordinary least squares and generalized least squares estimators, are either Bayes linear estimators or their limit points. In this paper, we discuss the statistical properties and optimality of Bayes linear estimators. First, we explore properties of Bayes linear estimators such as linear sufficiency and linear completeness. Second, we derive necessary and sufficient conditions under which two Bayes linear estimators coincide. In particular, several examples, including Rao's mixed-effects model and the general linear model with a spatial error process, demonstrate that our results can lead to a more efficient estimation procedure. Finally, we establish equivalence conditions for the equality of residual sums of squares when Bayes linear estimators are considered.