Parameter Estimation in Recurrent Tumor Evolution with Finite Carrying Capacity

TL;DR

This work develops a density-dependent, carrying-capacity–constrained two-type branching model to study tumor relapse under therapy, incorporating a rare mutation from sensitive to resistant cells and a pre-existing resistant clone. It proves a functional law of large numbers for the stochastic system up to tumor recurrence, derives precise asymptotics for recurrence time and clonal diversity, and constructs statistically consistent estimators for $\lambda_0$, $\lambda_1$, $\alpha$, and $\beta$ from observable quantities. Theoretical results are complemented by Gillespie simulations that confirm convergence of stochastic trajectories to deterministic limits and validate estimator performance under plausible biological regimes, including robustness to parameter variation and carrying-capacity scaling. The framework enables inference of key evolutionary parameters from longitudinal clonal data and medical imaging, supporting evolution-informed strategies to delay relapse and optimize therapy.

Abstract

In this work, we investigate the population dynamics of tumor cells under therapeutic pressure. Although drug treatment initially induces a reduction in tumor burden, treatment failure frequently occurs over time due to the emergence of drug resistance, ultimately leading to cancer recurrence. To model this process, we employ a two-type branching process with state-dependent growth rates. The model assumes an initial tumor population composed predominantly of drug-sensitive cells, with a small subpopulation of resistant cells. Sensitive cells may acquire resistance through mutation, which is coupled to a change in cellular fitness. Furthermore, the growth rates of resistant cells are modulated by the overall tumor burden. Using stochastic differential equation techniques, we establish a functional law of large numbers for the scaled populations of sensitive cells, resistant cells, and the initial resistant clone. We then define the stochastic recurrence time as the first time the total tumor population regrows to its initial size following treatment. For this recurrence time, as well as for measures of clonal diversity and the size of the largest resistant clone at recurrence, we derive corresponding law of large number limits. These asymptotic results provide a theoretical foundation for constructing statistically consistent estimators for key biological parameters, including the cellular growth rates, the mutation rate, and the initial fraction of resistant cells.

Figures (4)

• Figure 1: Simulated tumor dynamics under therapeutic pressure for increasing system sizes $n = 10^3, 10^4, 10^5, 10^6$. Parameter values: $\alpha = 0.8$, $\beta = 0.5$, $\lambda_0 = -0.5$, $\lambda_1 = 0.5$, $k = 3$. Solid lines represent stochastic trajectories ($Z_0$: sensitive cells, $Z_1$: total resistant cells, $Z_{\beta}$: pre-existing resistant clone). Dashed lines show corresponding scaled deterministic solutions ($K y_0, K y_1, K y_{\beta}$). As $n \to \infty$, stochastic fluctuations diminish and trajectories converge uniformly to their deterministic limits.
• Figure 2: Relative error of parameter estimators for increasing system sizes $n = 10^3, 10^4, 10^5, 10^6, 10^7$. Parameter values: $\alpha = 0.8$, $\beta = 0.5$, $\lambda_0 = -0.5$, $\lambda_1 = 0.5$, $k = 3$. Solid lines: mean relative error. Shaded areas: $\pm 1$ standard deviation.
• Figure 3: Relative errors of estimators across randomized parameter settings. Parameters are sampled from: $\lambda_0 \in (-0.9, -0.1)$, $\lambda_1 \in (0.1, 0.9)$, $\alpha \in (0.5, 0.9)$, $\beta \in (0.1, 0.9)$, and $k \in (1.5, 6.5)$. The sample size is fixed at $n = 5 \times 10^6$. Blue dots represent individual simulation runs; histogram bars represent bin-wise mean errors; the red line represents the overall mean error.
• Figure 4: Effect of the carrying capacity scaling factor $k$ on estimator performance. The vertical axis displays the average relative error across all four estimators $(\hat{\alpha}, \hat{\beta}, \hat{\lambda}_0, \hat{\lambda}_1)$. Blue dots represent individual simulation runs; histogram bars represent bin-wise mean errors; the red line represents the overall mean error.

Theorems & Definitions (24)

• Proposition 1
• proof
• Proposition 2
• proof
• Theorem 1
• proof
• Proposition 3
• proof
• Proposition 4
• proof