An Adaptive Control Approach to Treatment Selection for Substance Use Disorders

TL;DR

The paper addresses adaptive treatment selection in SUDs by modeling patient engagement as a latent state and treatment burden as an action with adherence-dependent rewards. It develops a fully personalized, discrete-time engagement/adherence model and casts the provider’s decision as a partial-information stochastic control problem, solved with certainty-equivalence dynamic programming. Unknown patient parameters are estimated via MLE/MAP with consistency guarantees, and two VI-based control algorithms are proposed. Computational results on synthetic patients show the approach can outperform heuristics and reveal nontrivial policy structures, supporting scalable, personalized digital therapeutics for continuing care in SUDs.

Abstract

Despite the massive costs and widespread harms of substance use, most individuals with substance use disorders (SUDs) receive no treatment at all. Digital therapeutics platforms are an emerging low-cost and low-barrier means of extending treatment to those who need it. While there is a growing body of research focused on how treatment providers can identify which patients need SUD support (or when they need it), there is very little work that addresses how providers should select treatments that are most appropriate for a given patient. Because SUD treatment involves months or years of voluntary compliance from the patient, treatment adherence is a critical consideration for the treatment provider. In this paper we focus on algorithms that a treatment provider can use to match the burden-level of proposed treatments to the time-varying engagement state of the patient to promote adherence. We propose structured models for a patient's engagement over time and their treatment adherence decisions. Using these models we pose a stochastic control formulation of the treatment-provider's burden selection problem. We propose an adaptive control approach that estimates unknown patient parameters as new data are observed. We show that these estimates are consistent and propose algorithms that use these estimates to make appropriate treatment recommendations.

Figures (2)

• Figure 1: Different optimal policy structures obtained by varying a single parameter ($c_\ell = 0.1, 0.2, 0.3$). Policies were obtained by value iteration. The parameters $(b,\lambda_\ell, \lambda_h, c_h, \gamma_\ell, \gamma_h,\alpha)=(0.9,0.7,0.8,1,0.5,1,0.95)$ were otherwise identical across the three models.
• Figure 2: Performance of the tested algorithms, separated for Patient 1 (left) and Patient 2 (right). For each patient, the left three plots show the optimal policies of each algorithm after trajectories of increasing length. Policy curves are the median values across replications. For each patient, the right three plots show performance summaries for 720-day trajectories, given as the mean for each algorithm across replications. Vertically stacked plots share x-axes.

Theorems & Definitions (11)

• Proposition 1
• proof
• Remark 1
• Remark 2
• Proposition 2
• proof
• Definition 1
• Proposition 3
• proof
• Corollary 1