Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MiWaves Reinforcement Learning Algorithm

Susobhan Ghosh, Yongyi Guo, Pei-Yao Hung, Lara Coughlin, Erin Bonar, Inbal Nahum-Shani, Maureen Walton, Susan Murphy

TL;DR

A comprehensive overview of the MiWaves RL algorithm's design is presented, including key decisions and experimental outcomes, which was deployed in a clinical trial from March to May 2024.

Abstract

The escalating prevalence of cannabis use poses a significant public health challenge globally. In the U.S., cannabis use is more prevalent among emerging adults (EAs) (ages 18-25) than any other age group, with legalization in the multiple states contributing to a public perception that cannabis is less risky than in prior decades. To address this growing concern, we developed MiWaves, a reinforcement learning (RL) algorithm designed to optimize the delivery of personalized intervention prompts to reduce cannabis use among EAs. MiWaves leverages domain expertise and prior data to tailor the likelihood of delivery of intervention messages. This paper presents a comprehensive overview of the algorithm's design, including key decisions and experimental outcomes. The finalized MiWaves RL algorithm was deployed in a clinical trial from March to May 2024.

MiWaves Reinforcement Learning Algorithm

TL;DR

A comprehensive overview of the MiWaves RL algorithm's design is presented, including key decisions and experimental outcomes, which was deployed in a clinical trial from March to May 2024.

Abstract

The escalating prevalence of cannabis use poses a significant public health challenge globally. In the U.S., cannabis use is more prevalent among emerging adults (EAs) (ages 18-25) than any other age group, with legalization in the multiple states contributing to a public perception that cannabis is less risky than in prior decades. To address this growing concern, we developed MiWaves, a reinforcement learning (RL) algorithm designed to optimize the delivery of personalized intervention prompts to reduce cannabis use among EAs. MiWaves leverages domain expertise and prior data to tailor the likelihood of delivery of intervention messages. This paper presents a comprehensive overview of the algorithm's design, including key decisions and experimental outcomes. The finalized MiWaves RL algorithm was deployed in a clinical trial from March to May 2024.
Paper Structure (36 sections, 19 equations, 7 figures, 3 tables, 1 algorithm)

This paper contains 36 sections, 19 equations, 7 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Formulating the RL problem
  3. Code
  4. Reinforcement Learning Framework
  5. Online Learning Algorithm
  6. Reward Approximating Function
  7. Online model update procedure
  8. Posterior update
  9. Hyper-parameter update
  10. Action Selection Procedure
  11. Prior data and MiWaves Simulation Testbed
  12. SARA vs MiWaves
  13. Data Extraction
  14. Data Cleaning
  15. Reward computation
  16. ...and 21 more sections

Figures (7)

  • Figure 1: Posterior sampling allocation functions
  • Figure 4: Comparison of log loss between the two models across all participants
  • Figure 5: Bar plot of coefficients of features in the MLR participant models relative to coefficients of class 0, across all $N=42$ participants.
  • Figure 7: GEE Results
  • Figure 8: Boxplot of Average reward per participant across 500 simulations. $120$ participants per simulation for $T=60$ decision times.
  • ...and 2 more figures