Assessing Surrogate Heterogeneity in Real World Data Using Meta-Learners

TL;DR

Knowlton and Parast introduce a framework to quantify heterogeneous surrogate strength in observational data by decomposing the treatment effect into covariate-specific components and defining a covariate-dependent PTE, RS(x). They implement this with meta-learners (linear, GAM, and regression forests) using sample-splitting and bootstrap to produce uncertainty and an algorithm to identify individuals for whom the surrogate is sufficiently strong. Through simulations across settings with and without heterogeneity and an NHANES-based example (HbA1c as a surrogate for fasting glucose under obesity), they demonstrate that surrogate strength can vary with covariates and that their approach supports personalized decision-making about surrogate use. The work emphasizes the role of strong, though untestable, causal assumptions, discusses practical considerations for estimator choice and thresholding, and provides software to enable applied adoption.

Abstract

Surrogate markers are most commonly studied within the context of randomized clinical trials. However, the need for alternative outcomes extends beyond these settings and may be more pronounced in real-world public health and social science research, where randomized trials are often impractical. Research on identifying surrogates in real-world non-randomized data is scarce, as available statistical approaches for evaluating surrogate markers tend to rely on the assumption that treatment is randomized. While the few methods that allow for non-randomized treatment/exposure appropriately handle confounding individual characteristics, they do not offer a way to examine surrogate heterogeneity with respect to patient characteristics. In this paper, we propose a framework to assess surrogate heterogeneity in real-world, i.e., non-randomized, data and implement this framework using various meta-learners. Our approach allows us to quantify heterogeneity in surrogate strength with respect to patient characteristics while accommodating confounders through the use of flexible, off-the-shelf machine learning methods. In addition, we use our framework to identify individuals for whom the surrogate is a valid replacement of the primary outcome. We examine the performance of our methods via a simulation study and application to examine heterogeneity in the surrogacy of hemoglobin A1c as a surrogate for fasting plasma glucose.

Figures (5)

• Figure 1: Estimated $R_S({\bf x})$ (solid lines) vs. true $R_S({\bf x})$ (dashed lines) plotted against $X_1$, the baseline covariate featuring heterogeneous surrogate strength in our simulations, with pointwise confidence bands (grey shading) obtained using bootstrapping.
• Figure 2: Estimation results for the NHANES survey data, evaluating the strength of HbA1c as a surrogate marker for plasma fasting glucose, when the exposure is obesity status; subfigures show the distribution of PTE estimates (top left panel) and PTE estimates by cholesterol (top right), age (bottom left), and sex (bottom right).
• Figure A1: Estimated $R_S({\bf x})$ (solid lines) vs. true $R_S({\bf x})$ (dashed lines) plotted against $X_1$ for Setting 4, which features no heterogeneity in the PTE. Confidence bands (grey shading) obtained using bootstrapping.
• Figure A2: Estimation results for the NHANES survey data using GAMs as the base learners, evaluating the strength of HbA1c as a surrogate marker for plasma fasting glucose, when the exposure is obesity status; subfigures show the distribution of PTE estimates (top left panel) and PTE estimates by cholesterol (top right), age (bottom left), and sex (bottom right).
• Figure A3: Estimation results for the NHANES survey data using regression forests as the base learners, evaluating the strength of HbA1c as a surrogate marker for plasma fasting glucose, when the exposure is obesity status; subfigures show the distribution of PTE estimates (top left panel) and PTE estimates by cholesterol (top right), age (bottom left), and sex (bottom right).