A Causal Machine Learning Framework for Treatment Personalization in Clinical Trials: Application to Ulcerative Colitis

TL;DR

The paper addresses the gap between detecting heterogeneous treatment effects in randomized trials and achieving improved patient outcomes through personalized decisions. It introduces a modular causal ML pipeline that uses X-learner CATE estimation, permutation importance, BLP testing, and doubly robust policy evaluation, applying it to the UNIFI ulcerative colitis maintenance trial. The key finding is that week-8 endoscopic features, while statistically associated with heterogeneity, do not improve and can even degrade policy performance for treatment assignment, whereas clinical markers like calprotectin, age, and CRP better capture decision-relevant variation. This work highlights the necessity of policy-level evaluation in causal ML analyses of trials and provides a diagnostic strategy to distinguish prognostic signals from true effect modifiers. The practical impact is a more reliable framework for assessing when personalized treatment decisions in trials will translate into better patient outcomes.

Abstract

Randomized controlled trials estimate average treatment effects, but treatment response heterogeneity motivates personalized approaches. A critical question is whether statistically detectable heterogeneity translates into improved treatment decisions -- these are distinct questions that can yield contradictory answers. We present a modular causal machine learning framework that evaluates each question separately: permutation importance identifies which features predict heterogeneity, best linear predictor (BLP) testing assesses statistical significance, and doubly robust policy evaluation measures whether acting on the heterogeneity improves patient outcomes. We apply this framework to patient-level data from the UNIFI maintenance trial of ustekinumab in ulcerative colitis, comparing placebo, standard-dose ustekinumab every 12 weeks, and dose-intensified ustekinumab every 8 weeks, using cross-fitted X-learner models with baseline demographics, medication history, week-8 clinical scores, laboratory biomarkers, and video-derived endoscopic features. BLP testing identified strong associations between endoscopic features and treatment effect heterogeneity for ustekinumab versus placebo, yet doubly robust policy evaluation showed no improvement in expected remission from incorporating endoscopic features, and out-of-fold multi-arm evaluation showed worse performance. Diagnostic comparison of prognostic contribution against policy value revealed that endoscopic scores behaved as disease severity markers -- improving outcome prediction in untreated patients but adding noise to treatment selection -- while clinical variables (fecal calprotectin, age, CRP) captured the decision-relevant variation. These results demonstrate that causal machine learning applications to clinical trials should include policy-level evaluation alongside heterogeneity testing.

Figures (2)

• Figure 1: Permutation importance for predicting heterogeneous treatment effects (UST vs Placebo). Features are color-coded by category: blue indicates clinical/laboratory features, red indicates endoscopic features. Error bars represent 95% confidence intervals. Clinical features, particularly inflammatory biomarkers (fecal calprotectin, CRP) and patient age, show the highest importance for identifying treatment effect heterogeneity.
• Figure 2: Forest plot comparing doubly robust policy values across treatment comparisons. Each point represents the expected remission rate under the learned policy, with 95% confidence intervals. Blue points indicate policies using all features (including endoscopy); purple points indicate clinical-only policies. The multi-arm out-of-fold comparison shows notably lower performance for all-features policies, suggesting overfitting when endoscopic features are included.