Medical follow-up optimization: A Monte-Carlo planning strategy

TL;DR

This work proposes to model cancer evolution with a Piecewise Deterministic Markov Process where patients alternate between remission and relapse phases with disease-specific tumor evolution, taking advantage of the nearly-deterministic nature of cancer evolution.

Abstract

Designing patient-specific follow-up strategy is a crucial step towards personalized medicine in cancer. Tools to help doctors deciding on treatment allocation together with next visit date, based on patient preferences and medical observations, would be particularly beneficial. Such tools should be based on realistic models of disease progress under the impact of medical treatments, involve the design of (multi-)objective functions that a treatment strategy should optimize along the patient's medical journey, and include efficient resolution algorithms to optimize personalized follow-up by taking the patient's history and preferences into account. We propose to model cancer evolution with a Piecewise Deterministic Markov Process where patients alternate between remission and relapse phases with disease-specific tumor evolution. This model is controlled via the online optimization of a long-term cost function accounting for treatment side-effects, hospital visits burden and disease impact on the quality of life. Optimization is based on noisy measurements of blood markers at visit dates. We leverage the Partially-Observed Monte-Carlo Planning algorithm to solve this continuous-time, continuous-state problem, taking advantage of the nearly-deterministic nature of cancer evolution. We show that this approximate solution approach of the exact model performs better than the counterpart exact resolution of the discrete model, while allowing for more versatility in the cost function model.

Figures (5)

• Figure 1: Example of patient follow-up data, PDMP model. a) Marker values are measured at each patient visits over a certain period of time. Data from the Intergroupe Francophone du Myélome 2009 clinical trial, courtesy of the Centre de Recherche en Cancérologie de Toulouse. b) PDMP model, representation of the marker level of a patient. The risk function $\lambda$ controls the time to relapse, while parameters $v$ and $v'$ control the aggressiveness of the disease and the efficiency of the treatment respectively.
• Figure 2: Practice of patient follow-up. a) At each new visit the patient has a new marker measurement, and the practitioner receives a new observation $\omega_n=(y_n,t_n)$. b) The filter is updated with the new observation, either through particle rejection sampling (particle filter) or via a recursion formula (conditional filter). c) The decision tree is partially explored via simulation through an adapted POMCP algorithm using the updated filter. The algorithm returns the optimal decision $d_n$, combination of a time to next visit (defining $t_{n+1}$) and treatment to allocate (influencing $y_{n+1}$).
• Figure 3: Impact of POMCP parameters on the estimated value function. Top left: increasing the number of simulations for filters with $500$ initial states improves the average trajectory costs. Top right: increasing the number of atoms in the filter improves the performance of the particle filter but not the conditional filter.
• Figure 4: a) Radar plot comparing performances of 4 solution strategies on death rate, progression-free survival (PFS), time spent under treatment, average number of visits per patient, and average trajectory cost. An optimal strategy would be the inner-circle. b) Barplot of trajectory cost for 500 simulations under three main strategies: POMCP with particle filter, POMCP with conditional filter and Dynamic Programming on discretized processes.
• Figure 5: Risk and Density functions for standard relapse from remission condition to disease $b$ condition (similar shapes for standard relapse to disease $a$).