Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximate Cross-validated Mean Estimates for Bayesian Hierarchical Regression Models

Amy X. Zhang, Le Bao, Changcheng Li, Michael J. Daniels

TL;DR

A novel procedure for obtaining cross-validated predictive estimates for Bayesian hierarchical regression models (BHRMs) is introduced, which circumvents the need to rerun computationally costly estimation methods for each cross-validation fold and makes CV more feasible for large BHRMs.

Abstract

We introduce a novel procedure for obtaining cross-validated predictive estimates for Bayesian hierarchical regression models (BHRMs). Bayesian hierarchical models are popular for their ability to model complex dependence structures and provide probabilistic uncertainty estimates, but can be computationally expensive to run. Cross-validation (CV) is therefore not a common practice to evaluate the predictive performance of BHRMs. Our method circumvents the need to re-run computationally costly estimation methods for each cross-validation fold and makes CV more feasible for large BHRMs. By conditioning on the variance-covariance parameters, we shift the CV problem from probability-based sampling to a simple and familiar optimization problem. In many cases, this produces estimates which are equivalent to full CV. We provide theoretical results and demonstrate its efficacy on publicly available data and in simulations.

Approximate Cross-validated Mean Estimates for Bayesian Hierarchical Regression Models

TL;DR

A novel procedure for obtaining cross-validated predictive estimates for Bayesian hierarchical regression models (BHRMs) is introduced, which circumvents the need to rerun computationally costly estimation methods for each cross-validation fold and makes CV more feasible for large BHRMs.

Abstract

We introduce a novel procedure for obtaining cross-validated predictive estimates for Bayesian hierarchical regression models (BHRMs). Bayesian hierarchical models are popular for their ability to model complex dependence structures and provide probabilistic uncertainty estimates, but can be computationally expensive to run. Cross-validation (CV) is therefore not a common practice to evaluate the predictive performance of BHRMs. Our method circumvents the need to re-run computationally costly estimation methods for each cross-validation fold and makes CV more feasible for large BHRMs. By conditioning on the variance-covariance parameters, we shift the CV problem from probability-based sampling to a simple and familiar optimization problem. In many cases, this produces estimates which are equivalent to full CV. We provide theoretical results and demonstrate its efficacy on publicly available data and in simulations.

Paper Structure

This paper contains 13 sections, 2 theorems, 13 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Approximate cross-validation estimates using plug-in estimators (AXE)
  3. AXE for linear mixed models
  4. AXE for generalized linear mixed models
  5. Leave-one-cluster-out cross-validation (LCO-CV)
  6. Asymptotic behavior under LCO-CV
  7. Finite sample performance
  8. Existing CV approximation methods
  9. Results
  10. LRR
  11. Point-by-point comparisons
  12. Computation time
  13. Discussion

Key Result

Theorem 2.1

Let response vector $Y \in I\!\!R^N$ of a hierarchical linear regression follow a normal distribution as in (eqn:normlinreg) and (eqn:hyper) and define $\theta$ as in (eqn:def_mutheta). The data are partitioned into $J$ CV folds based on $\theta$, where all data informing $\theta_j$ correspond to th where $I$ is the identity matrix, $\|.\|$ is the operator norm of a matrix defined as $\|A\|=\sqrt{

Figures (2)

  • Figure 1: Line plots of the proportion of CV folds $j$ with $|\text{LRR}|_j \le x$, for $x \in [0, \log(2)]$ on the x-axis. We refer to each line as an LRR percentage curve. Curves are colored and shaped based on the method used and are truncated at $\log(2) \approx 0.7$. Ghost is only applicable to LMMs in examples A, B and C.
  • Figure 2: Scatter plots comparing the LCO approximation to ground-truth MCV estimate for each data point, model, and data set. Panels in row A compare the AXE approximation $\hat{Y}_{ji}^{\text{AXE}}$ against the MCV estimate for $E(Y_{ji} | Y_{-\mathpzc{s}_j})$. Panels in row B add points with GHOST (pink triangle) and iIS (green square) approximations whenever applicable, along with AXE (black circle). Each point in a grid represents one point in the corresponding data set(s) and model(s).

Theorems & Definitions (9)

  • Theorem 2.1
  • Remark 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Remark 5
  • Remark 6
  • Remark 7
  • Corollary 2.2