Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Off-Policy Evaluation and Learning for Survival Outcomes under Censoring

Kohsuke Kubota, Mitsuhiro Takahashi, Yuta Saito

Abstract

Optimizing survival outcomes, such as patient survival or customer retention, is a critical objective in data-driven decision-making. Off-Policy Evaluation~(OPE) provides a powerful framework for assessing such decision-making policies using logged data alone, without the need for costly or risky online experiments in high-stakes applications. However, typical estimators are not designed to handle right-censored survival outcomes, as they ignore unobserved survival times beyond the censoring time, leading to systematic underestimation of the true policy performance. To address this issue, we propose a novel framework for OPE and Off-Policy Learning~(OPL) tailored for survival outcomes under censoring. Specifically, we introduce IPCW-IPS and IPCW-DR, which employ the Inverse Probability of Censoring Weighting technique to explicitly deal with censoring bias. We theoretically establish that our estimators are unbiased and that IPCW-DR achieves double robustness, ensuring consistency if either the propensity score or the outcome model is correct. Furthermore, we extend this framework to constrained OPL to optimize policy value under budget constraints. We demonstrate the effectiveness of our proposed methods through simulation studies and illustrate their practical impacts using public real-world data for both evaluation and learning tasks.

Off-Policy Evaluation and Learning for Survival Outcomes under Censoring

Abstract

Optimizing survival outcomes, such as patient survival or customer retention, is a critical objective in data-driven decision-making. Off-Policy Evaluation~(OPE) provides a powerful framework for assessing such decision-making policies using logged data alone, without the need for costly or risky online experiments in high-stakes applications. However, typical estimators are not designed to handle right-censored survival outcomes, as they ignore unobserved survival times beyond the censoring time, leading to systematic underestimation of the true policy performance. To address this issue, we propose a novel framework for OPE and Off-Policy Learning~(OPL) tailored for survival outcomes under censoring. Specifically, we introduce IPCW-IPS and IPCW-DR, which employ the Inverse Probability of Censoring Weighting technique to explicitly deal with censoring bias. We theoretically establish that our estimators are unbiased and that IPCW-DR achieves double robustness, ensuring consistency if either the propensity score or the outcome model is correct. Furthermore, we extend this framework to constrained OPL to optimize policy value under budget constraints. We demonstrate the effectiveness of our proposed methods through simulation studies and illustrate their practical impacts using public real-world data for both evaluation and learning tasks.
Paper Structure (48 sections, 3 theorems, 49 equations, 4 figures, 2 tables)

This paper contains 48 sections, 3 theorems, 49 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem Formulation
  4. Policy Value Definition.
  5. Assumptions.
  6. Naive OPE under Censoring
  7. Proposed Approach: Censoring-Aware Off-Policy Evaluation and Learning
  8. Unbiased Estimation of Survival Policy Value
  9. Off-Policy Learning for Survival Maximization
  10. Extension to Constrained Off-Policy Learning.
  11. Simulation Studies
  12. Data Generating Process.
  13. OPE Performance Comparison.
  14. OPL Performance Comparison.
  15. Simulation Results.
  16. ...and 33 more sections

Key Result

Proposition 2.3

Under the presence of censoring (that is, $P(L > C) > 0$) and Assumptions ass:common_support and ass:independent_censoring, the naive IPS and DR estimators are biased as where $G(t \mid x, a) \coloneq P(C > t \mid x, a)$ is the censoring survival function.

Figures (4)

  • Figure 1: Illustration of the underestimation bias inherent in naive OPE. While the true event time for User B is unobserved due to censoring, naive estimators treat the censoring time as the event time, leading to a systematic underestimation of the survival time.
  • Figure 2: OPE performance comparison across key experimental factors. The figure illustrates how MSE changes as we vary (a) the logged data size $n$, (b) the censoring rate $\rho_1$, and (c) the divergence between the logging and evaluation policies $\epsilon$.
  • Figure 3: OPL performance comparison. The improvement ratio $V^\tau(\hat{\pi}) / V^\tau(\pi_0)$ quantifies how effectively each method learns a superior policy compared to the logging policy optimalities.
  • Figure 4: OPL performance comparison under varying $\beta$. The improvement ratio $V^\tau(\hat{\pi}) / V^\tau(\pi_0)$ quantifies how effectively each method learns a superior policy compared to the logging policy optimalities.

Theorems & Definitions (9)

  • Proposition 2.3
  • Theorem 3.1
  • Theorem 3.2
  • proof
  • proof
  • proof
  • proof
  • proof
  • proof