Rethinking the Win Ratio: A Causal Framework for Hierarchical Outcome Analysis
Mathieu Even, Julie Josse
TL;DR
The paper tackles the challenge of causal inference with hierarchical, multivariate outcomes by embedding Win Ratio and Generalized Pairwise Comparisons in a formal potential-outcomes framework. It reveals that the estimand depends on how treated-control pairs are formed, showing that traditional complete pairings can yield misleading conclusions in heterogeneous populations. To address this, the authors introduce an identifiable, individual-level estimand $ au_igstar$ and establish that Nearest Neighbor pairings consistently estimate it in randomized settings; they also extend to observational data via IPW and a distributional-regression-based approach with a doubly robust variant. Through synthetic experiments and the CRASH-3 trial, they demonstrate that their methods can provide more robust and sometimes drastically different treatment recommendations than traditional approaches, highlighting the importance of targeting the appropriate estimand for valid causal interpretation and application to real-world data.
Abstract
Quantifying causal effects in the presence of complex and multivariate outcomes is a key challenge to evaluate treatment effects. For hierarchical multivarariates outcomes, the FDA recommends the Win Ratio and Generalized Pairwise Comparisons approaches. However, as far as we know, these empirical methods lack causal or statistical foundations to justify their broader use in recent studies. To address this gap, we establish causal foundations for hierarchical comparison methods. We define related causal effect measures, and highlight that depending on the methodology used to compute Win Ratios or Net Benefits of treatments, the causal estimand targeted can be different, as proved by our consistency results. Quite dramatically, it appears that the causal estimand related to the historical estimation approach can yield reversed and incorrect treatment recommendations in heterogeneous populations, as we illustrate through striking examples. In order to compensate for this fallacy, we introduce a novel, individual-level yet identifiable causal effect measure that better approximates the ideal, non-identifiable individual-level estimand. We prove that computing Win Ratio or Net Benefits using a Nearest Neighbor pairing approach between treated and controlled patients, an approach that can be seen as an extreme form of stratification, leads to estimating this new causal estimand measure. We extend our methods to observational settings via propensity weighting, distributional regression to address the curse of dimensionality, and a doubly robust framework. We prove the consistency of our methods, and the double robustness of our augmented estimator. Finally, we validate our approach using synthetic data and on CRASH-3, a major clinical trial focused on assessing the effects of tranexamic acid in patients with traumatic brain injury.