Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to Call: A Field Trial of a Collaborative Bandit Algorithm for Improved Message Delivery in Mobile Maternal Health

Arpan Dasgupta, Mizhaan Maniyar, Awadhesh Srivastava, Sanat Kumar, Amrita Mahale, Aparna Hegde, Arun Suggala, Karthikeyan Shanmugam, Aparna Taneja, Milind Tambe

TL;DR

This study addresses the problem of missed mHealth calls in Kilkari by field-testing a collaborative bandit algorithm that learns mothers’ preferred call times. The approach, based on a Phased Matrix Completion method with shared structure, was deployed with approximately $6500$ participants in Odisha and compared to the baseline random calling strategy. The results show a significant improvement in call pick-up rates for the treatment group, especially in the Mid Tier, and an off-policy evaluation provides context forExploration needs in real-world deployment. Overall, the work demonstrates that personalized scheduling via collaborative bandits can meaningfully enhance the reach and impact of mobile maternal health messaging at scale.

Abstract

Mobile health (mHealth) programs utilize automated voice messages to deliver health information, particularly targeting underserved communities, demonstrating the effectiveness of using mobile technology to disseminate crucial health information to these populations, improving health outcomes through increased awareness and behavioral change. India's Kilkari program delivers vital maternal health information via weekly voice calls to millions of mothers. However, the current random call scheduling often results in missed calls and reduced message delivery. This study presents a field trial of a collaborative bandit algorithm designed to optimize call timing by learning individual mothers' preferred call times. We deployed the algorithm with around $6500$ Kilkari participants as a pilot study, comparing its performance to the baseline random calling approach. Our results demonstrate a statistically significant improvement in call pick-up rates with the bandit algorithm, indicating its potential to enhance message delivery and impact millions of mothers across India. This research highlights the efficacy of personalized scheduling in mobile health interventions and underscores the potential of machine learning to improve maternal health outreach at scale.

Learning to Call: A Field Trial of a Collaborative Bandit Algorithm for Improved Message Delivery in Mobile Maternal Health

TL;DR

This study addresses the problem of missed mHealth calls in Kilkari by field-testing a collaborative bandit algorithm that learns mothers’ preferred call times. The approach, based on a Phased Matrix Completion method with shared structure, was deployed with approximately participants in Odisha and compared to the baseline random calling strategy. The results show a significant improvement in call pick-up rates for the treatment group, especially in the Mid Tier, and an off-policy evaluation provides context forExploration needs in real-world deployment. Overall, the work demonstrates that personalized scheduling via collaborative bandits can meaningfully enhance the reach and impact of mobile maternal health messaging at scale.

Abstract

Mobile health (mHealth) programs utilize automated voice messages to deliver health information, particularly targeting underserved communities, demonstrating the effectiveness of using mobile technology to disseminate crucial health information to these populations, improving health outcomes through increased awareness and behavioral change. India's Kilkari program delivers vital maternal health information via weekly voice calls to millions of mothers. However, the current random call scheduling often results in missed calls and reduced message delivery. This study presents a field trial of a collaborative bandit algorithm designed to optimize call timing by learning individual mothers' preferred call times. We deployed the algorithm with around Kilkari participants as a pilot study, comparing its performance to the baseline random calling approach. Our results demonstrate a statistically significant improvement in call pick-up rates with the bandit algorithm, indicating its potential to enhance message delivery and impact millions of mothers across India. This research highlights the efficacy of personalized scheduling in mobile health interventions and underscores the potential of machine learning to improve maternal health outreach at scale.

Paper Structure

This paper contains 18 sections, 7 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Kilkari
  5. Collaborative Bandits
  6. Experimental Design
  7. Randomization
  8. Trial Phases
  9. Data Collection
  10. Outcome Measure
  11. Preliminaries
  12. Results
  13. Call Volumes
  14. Call Pick-up Rates
  15. Time slot wise analysis
  16. ...and 3 more sections

Figures (2)

  • Figure 1: An example poster image from the official X account of the MOFHW. Link https://x.com/MoHFW_INDIA/status/979600610952654848/photo/1.
  • Figure 2: The above bars represent the fraction of unique calls, i.e. the first call recommended by the algorithm without considering re-attempts. These give a truer representation of the call recommendation distribution or policy. (a) despite the distribution having a notable dip in slot 5 and 6, we still observe good pick-up success rates in slot 6 from Table \ref{['tab:timeslot_pickup_rates']}, and for (b) the distribution is almost uniform as expected.