Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A belief-state restless bandit model for treatment adherence: Whittle indexability via partial conservation laws

José Niño-Mora, Ángel Pellitero García

TL;DR

The work addresses capacity-limited outreach for treatment adherence by modeling patients as belief-state restless bandits with reset-type interventions. It develops a PCL-based verification framework that yields a closed-form Whittle index and an analytic Lagrangian relaxation, enabling efficient dual-bound computation in large populations. The authors provide explicit threshold maps, analyze parameter sensitivities (lapse $p$ and recovery $q$), and demonstrate, in a large-scale two-type numerical study, that Whittle’s index policy often matches or surpasses myopic policies and baselines, with notable gains (up to ~26%) in tight-capacity scenarios. The results offer a scalable, verifiable approach to deploying index-based policies for healthcare adherence outreach, with practical implications for resource allocation and policy evaluation.

Abstract

We study capacity-constrained treatment-adherence outreach via a belief-state restless multi-armed bandit model where patients are a partially observed two-state (adherent/nonadherent) Markov processes and interventions induce reset-type belief dynamics. Using partial conservation laws (PCLs), we establish Whittle indexability of the single-patient problem and derive a closed-form Whittle (marginal productivity) index, together with closed-form reward/work performance metrics under threshold policies and an explicit optimal threshold map. This yields an analytic Lagrangian relaxation: the single-patient Lagrangian value is a piecewise-affine convex function of the intervention price, enabling efficient computation of multi-patient dual bounds and certified relative optimality gaps. We also analyze how the Whittle index depends on the lapse and spontaneous-recovery parameters, providing qualitative insights on intervention priorities. In a large-scale numerical study over heterogeneous two-type populations, we compare Whittle's index policy with a myopic index rule and simple baselines; while myopic is highly competitive on most instances, Whittle's policy yields substantial gains in tight-capacity regimes with a fragile minority, reaching up to about $26\%$ higher reward and markedly smaller relative optimality gaps.

A belief-state restless bandit model for treatment adherence: Whittle indexability via partial conservation laws

TL;DR

The work addresses capacity-limited outreach for treatment adherence by modeling patients as belief-state restless bandits with reset-type interventions. It develops a PCL-based verification framework that yields a closed-form Whittle index and an analytic Lagrangian relaxation, enabling efficient dual-bound computation in large populations. The authors provide explicit threshold maps, analyze parameter sensitivities (lapse and recovery ), and demonstrate, in a large-scale two-type numerical study, that Whittle’s index policy often matches or surpasses myopic policies and baselines, with notable gains (up to ~26%) in tight-capacity scenarios. The results offer a scalable, verifiable approach to deploying index-based policies for healthcare adherence outreach, with practical implications for resource allocation and policy evaluation.

Abstract

We study capacity-constrained treatment-adherence outreach via a belief-state restless multi-armed bandit model where patients are a partially observed two-state (adherent/nonadherent) Markov processes and interventions induce reset-type belief dynamics. Using partial conservation laws (PCLs), we establish Whittle indexability of the single-patient problem and derive a closed-form Whittle (marginal productivity) index, together with closed-form reward/work performance metrics under threshold policies and an explicit optimal threshold map. This yields an analytic Lagrangian relaxation: the single-patient Lagrangian value is a piecewise-affine convex function of the intervention price, enabling efficient computation of multi-patient dual bounds and certified relative optimality gaps. We also analyze how the Whittle index depends on the lapse and spontaneous-recovery parameters, providing qualitative insights on intervention priorities. In a large-scale numerical study over heterogeneous two-type populations, we compare Whittle's index policy with a myopic index rule and simple baselines; while myopic is highly competitive on most instances, Whittle's policy yields substantial gains in tight-capacity regimes with a fragile minority, reaching up to about higher reward and markedly smaller relative optimality gaps.
Paper Structure (41 sections, 22 theorems, 171 equations, 6 figures, 3 tables)

This paper contains 41 sections, 22 theorems, 171 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related work on belief-state adherence bandits and dual bounds
  4. Model description and RMABP formulation
  5. Indexability, dual bound, and Whittle index policy
  6. PCL-based verification theorem for threshold-indexability
  7. Verification of the PCL-indexability conditions
  8. Discounted reward and work metrics under threshold policies
  9. Discounted marginal metrics and condition (PCLI1)
  10. MP index and condition (PCLI2)
  11. Verifying (PCLI3) and threshold-indexability
  12. The optimal threshold map $z^*(\lambda)$
  13. Analytic Lagrangian relaxation and dual bound computation
  14. Population-averaged metrics under a uniform initial belief
  15. Single-patient piecewise-affine Lagrangian value
  16. ...and 26 more sections

Key Result

Theorem 1

Under (PCLI1--PCLI3), the model is threshold-indexable. Moreover, the Whittle index $w(x)$ equals the MP index $m(x)$, and the optimal threshold maps $z^*(\cdot)$ are the generalized inverses of $m(\cdot)$.

Figures (6)

  • Figure 1: Reward and work metrics as functions of the threshold $z$ for fixed belief $x$.
  • Figure 2: Marginal reward and work metrics as functions of the threshold $z$ for fixed $x$.
  • Figure 3: MP index $m(x)$ versus belief state $x$.
  • Figure 4: Optimal threshold $z^*(\lambda)$ versus intervention price $\lambda$.
  • Figure 5: Dependence of the MP index $m(x)$ on the lapse probability $p$ for a fixed $x$.
  • ...and 1 more figures

Theorems & Definitions (49)

  • Remark 1: Deterministic belief dynamics
  • Remark 2: Per-intervention costs
  • Theorem 1: nmmor20
  • Proposition 1
  • Proposition 2
  • Lemma 1: PCLI1
  • proof
  • Proposition 3
  • proof : Proof sketch
  • Remark 3
  • ...and 39 more