PyDTS: A Python Package for Discrete-Time Survival Analysis with Competing Risks and Optional Penalization

TL;DR

PyDTS tackles discrete-time survival data with competing risks by implementing two estimation frameworks: the Collapsed Log-Likelihood approach and the faster two-step method for semi-parametric logit-link models. It supports time-dependent covariates, regularized regression on the betas, and performance metrics such as cause-specific AUC and Brier score, along with a simulation module and comprehensive prediction routines for S(t|Z), λ_j(t|Z), Pr(T=t,J=j|Z), and F_j(t|Z). The package enables model evaluation via CV and grid-search for penalty tuning, and provides data regrouping utilities to handle sparse events. Demonstrated on the MIMIC-IV LOS case study, PyDTS shows competitive estimation accuracy and substantial speedups for large datasets, making discrete-time competing-risks analysis accessible in Python.

Abstract

Time-to-event (survival) analysis models the time until a pre-specified event occurs. When time is measured in discrete units or rounded into intervals, standard continuous-time models can yield biased estimators. In addition, the event of interest may belong to one of several mutually exclusive types, referred to as competing risks, where the occurrence of one event prevents the occurrence or observation of the others. PyDTS is an open-source Python package for analyzing discrete-time survival data with competing-risks. It provides regularized estimation methods, model evaluation metrics, variable screening tools, and a simulation module to support research and development.

Figures (10)

• Figure 1: Sampled covariates. a. The observed distribution of $Z_1$, b. The observed distribution of $Z_2|Z_1$. c. The distribution of $Z_3$.
• Figure 2: Events and censoring distribution of the simulated dataset with $n=50,000$ observations, $M=2$ event types, and $d=30$ time points. $\alpha_{1t} = -1 -0.3 \log t$, $\alpha_{2t} = -1.75 -0.15\log(t)$, $\beta^T_1 = -(\log 0.8, \log 3, \log 3, \log 2.5, \log 2)$, $\beta^T_{2} = -(\log 1, \log 3, \log 4, \log 3, \log 2)$ and censoring times are sampled from $\hbox{Uniform}\{1,...,d+1\}$.
• Figure 3: Results of one simulated dataset, $n=50,000$: Estimation results of lee_analysis_2018 and the two-step procedure of meir_gorfine_dtsp_2023 along with the true parameters' values. a: Results of $\alpha_{jt}$. True values of $\alpha_{1t}$ are in dashed blue line, estimates of Lee et al. and the two-step algorithm are in light blue circles and blue stars, respectively. Similarly, true values and estimates of $\alpha_{2t}$ are in green scale. Number of observed events at each time $t$ is shown in red and brown bars. b: Results of $\beta_{j}$. True values of $\beta_{1}$ are in light blue bars, estimates of Lee et al. and the two-step algorithm are in light blue right arrowhead and blue right triangle, respectively. Similarly, true values and estimates of $\beta_{2}$ are in green scale.
• Figure 4: Predictions for three individuals of the test dataset: ID=0 (blue), ID=1 (orange), and ID=2 (green). a. Estimated hazard function of event type $j=1$, $\widehat{\lambda}_{1}(t)$. b. Estimated hazard function of event type $j=2$, $\widehat{\lambda}_{1}(t|Z)$. c. Estimated probability of event type $j=1$, $\widehat{\Pr}(T=t,J=1|Z)$. d. Estimated probability for event type $j=2$, $\widehat{\Pr}(T=t,J=2|Z)$. e. Estimated CIF of event type $j=1$, $\widehat{F}_1(t|Z)$. f. Estimated CIF of event type $j=2$, $\widehat{F}_2(t|Z)$. g. Estimated overall survival function, $\widehat{S}(t|Z)$.
• Figure 5: Simulation results of lee_analysis_2018, the two-step algorithm of meir_gorfine_dtsp_2023 the true parameters' values based on 100 repetitions. a: Results of $\alpha_{jt}$. True values of $\alpha_{1t}$ are in dashed blue line, estimates of Lee et al. and the two-step algorithm are in light blue circles and blue stars, respectively. Similarly, true values and estimates of $\alpha_{2t}$ are in green scale. Number of observed events at each time $t$ is shown in red and brown bars. b: Results of $\beta_{j}$. True values of $\beta_{1}$ are in light blue bars, estimates of Lee et al. and the two-step algorithm are in light blue right arrowhead and blue right triangle, respectively. Similarly, true values and estimates of $\beta_{2}$ are in green scale.