Comparison of the Cox proportional hazards model and Random Survival Forest algorithm for predicting patient-specific survival probabilities in clinical trial data

TL;DR

This study addresses how to predict patient-specific survival in randomized trials by comparing the Cox proportional hazards model and Random Survival Forest (RSF) using a neutral, multi-faceted evaluation. It combines bootstrap validation and data simulations based on two reference RCT datasets (one without and one with treatment-covariate interactions) to assess discrimination, calibration, and overall performance across several RSF splitting rules. The findings show that relying solely on the discriminative $C$ index can be misleading; integrated measures like the Integrated Brier Score and calibration curves reveal that RSF often excels in scenarios with interactions or nonproportional hazards, while Cox-PH can outperform RSF in calibration and IBS under other settings. These results highlight the importance of comprehensive performance assessment in clinical trial prognosis, and suggest that selecting RSF splitting rules beyond the default can improve predictive accuracy in certain contexts.

Abstract

The Cox proportional hazards model is often used to analyze data from Randomized Controlled Trials (RCT) with time-to-event outcomes. Random survival forest (RSF) is a machine-learning algorithm known for its high predictive performance. We conduct a comprehensive neutral comparison study to compare the performance of Cox regression and RSF in various simulation scenarios based on two reference datasets from RCTs. The motivation is to identify settings in which one method is preferable over the other when comparing different aspects of performance using measures according to the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis) recommendations. Our results show that conclusions solely based on the C index, a performance measure that has been predominantly used in previous studies comparing predictive accuracy of the Cox-PH and RSF model based on real-world observational time-to-event data and that has been criticized by methodologists, may not be generalizable to other aspects of predictive performance. We found that measures of overall performance may generally give more reasonable results, and that the standard log-rank splitting rule used for the RSF may be outperformed by alternative splitting rules, in particular in nonproportional hazards settings. In our simulations, performance of the RSF suffers less in data with treatment-covariate interactions compared to data where these are absent. Performance of the Cox-PH model is affected by the violation of the proportional hazards assumption.

Figures (11)

• Figure 1: Bootstrap estimate $\hat{\theta}^{.632+}$ (95% confidence interval) of the $C$ index (right) and Integrated Brier score (left) for the RCT in primary biliary cirrhosis patients. Abbreviations: Cox-PH - Cox proportional hazards model, RSF - Random survival forest.
• Figure 2: Bootstrap estimate $\hat{\theta}^{.632+}$ (95% confidence interval) of the $C$ index (left) and Integrated Brier score (right) for the RCT in prostate cancer patients. Abbreviations: Cox-PH - Cox proportional hazards model, RSF - Random survival forest.
• Figure 3: $C$ index estimates in the simulation scenario assuming 30% censoring and a treatment effect of $\beta_{\text{\normalfont{treatment}}} = -0.4$ for the RCT in primary biliary cirrhosis (a) and in prostate cancer patients (b). Survival times are generated from a Weibull distribution with scale parameters estimated from the respective reference dataset, shape parameters ($\gamma$) vary in order to examine the impact of differing hazards, and the violation of the proportional hazards assumption. Results are shown for different total sample sizes $N$. Abbreviations: Cox-PH - Cox proportional hazards model, RSF - Random survival forest.
• Figure 4: $C$ index estimates in the simulation scenario assuming 60% censoring and a treatment effect of $\beta_{\text{\normalfont{treatment}}} = -0.4$ for the RCT in primary biliary cirrhosis (a) and in prostate cancer patients (b). Survival times are generated from a Weibull distribution with scale parameters estimated from the respective reference dataset, shape parameters ($\gamma$) vary in order to examine the impact of differing hazards, and the violation of the proportional hazards assumption. Results are shown for different total sample sizes $N$. Abbreviations: Cox-PH - Cox proportional hazards model, RSF - Random survival forest.
• Figure 5: Integrated Brier score (IBS) estimates in the simulation scenario assuming 30% censoring and a treatment effect of $\beta_{\text{\normalfont{treatment}}} = -0.4$ for the RCT in primary biliary cirrhosis (a) and in prostate cancer patients (b). Survival times are generated from a Weibull distribution with scale parameters estimated from the respective reference dataset, shape parameters ($\gamma$) vary in order to examine the impact of differing hazards, and the violation of the proportional hazards assumption. Results are shown for different total sample sizes $N$. Abbreviations: Cox-PH - Cox proportional hazards model, RSF - Random survival forest.