Fixed and Random Covariance Regression Analyses

TL;DR

The paper addresses covariance regression when the explanatory variables $X$ are random, proposing a complete estimation and model-assessment framework that extends beyond traditional Fixed-$X$ analyses. It develops two estimation strategies, quasi-maximum likelihood estimation and weighted least squares, proving consistency and asymptotic normality under various growing-dimension regimes and without relying on i.i.d. assumptions. A novel model-assessment theory for Random-$X$ covariance regression is introduced, including bias-variance decompositions, Mallows' Cp analogues, and cross-validation estimators (RCp and OCV) to quantify expected test errors under different data-generation settings. Through extensive simulations and an empirical stock-returns study, the authors demonstrate that randomness in $X$ can increase both bias and variance in expected test errors and show how the proposed RCp and OCV tools provide more reliable model evaluation and selection in practice. Overall, the work broadens the applicability of covariance-regression models to realistic, dependence-structured, high-dimensional settings and lays groundwork for future extensions to nonparametric and time-series contexts.

Abstract

Covariance regression analysis is an approach to linking the covariance of responses to a set of explanatory variables $X$, where $X$ can be a vector, matrix, or tensor. Most of the literature on this topic focuses on the "Fixed-$X$" setting and treats $X$ as nonrandom. By contrast, treating explanatory variables $X$ as random, namely the "Random-$X$" setting, is often more realistic in practice. This article aims to fill this gap in the literature on the estimation and model assessment theory for Random-$X$ covariance regression models. Specifically, we construct a new theoretical framework for studying the covariance estimators under the Random-$X$ setting, and we demonstrate that the quasi-maximum likelihood estimator and the weighted least squares estimator are both consistent and asymptotically normal. In addition, we develop pioneering work on the model assessment theory of covariance regression. In particular, we obtain the bias-variance decompositions for the expected test errors under both the Fixed-$X$ and Random-$X$ settings. We show that moving from a Fixed-$X$ to a Random-$X$ setting can increase both the bias and the variance in expected test errors. Subsequently, we propose estimators of the expected test errors under the Fixed-$X$ and Random-$X$ settings, which can be used to assess the performance of the competing covariance regression models. The proposed estimation and model assessment approaches are illustrated via extensive simulation experiments and an empirical study of stock returns in the US market.

Theorems & Definitions (15)

• Theorem 1
• Remark 1
• Theorem 2
• Remark 2
• Remark 3
• Remark 4
• Theorem 3
• Theorem 4
• Theorem 5
• Proposition 1