Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

User Engagement in Mobile Health Applications

Babaniyi Yusuf Olaniyi, Ana Fernández del Río, África Periáñez, Lauren Bellhouse

TL;DR

This paper tackles how to quantify and predict user engagement and churn in mobile health apps used by frontline healthcare workers in LMICs. It merges three complementary approaches—ECDF-based engagement indicators, a unified engagement score, and survival analysis using CSF and LTRC forests—to characterize individual and population-level engagement, with a focus on the Safe Delivery App in India and Ethiopia. The study demonstrates how churn can be detected earlier via ECDF methods, how a single engagement score can summarize multifaceted activity, and how time-to-churn predictions can be made despite censoring, offering practical pathways to tailor interventions for health workers. The framework holds potential to improve training and support through adaptive content and reminders, ultimately aiming to enhance patient care and health outcomes.

Abstract

Mobile health apps are revolutionizing the healthcare ecosystem by improving communication, efficiency, and quality of service. In low- and middle-income countries, they also play a unique role as a source of information about health outcomes and behaviors of patients and healthcare workers, while providing a suitable channel to deliver both personalized and collective policy interventions. We propose a framework to study user engagement with mobile health, focusing on healthcare workers and digital health apps designed to support them in resource-poor settings. The behavioral logs produced by these apps can be transformed into daily time series characterizing each user's activity. We use probabilistic and survival analysis to build multiple personalized measures of meaningful engagement, which could serve to tailor content and digital interventions suiting each health worker's specific needs. Special attention is given to the problem of detecting churn, understood as a marker of complete disengagement. We discuss the application of our methods to the Indian and Ethiopian users of the Safe Delivery App, a capacity-building tool for skilled birth attendants. This work represents an important step towards a full characterization of user engagement in mobile health applications, which can significantly enhance the abilities of health workers and, ultimately, save lives.

User Engagement in Mobile Health Applications

TL;DR

This paper tackles how to quantify and predict user engagement and churn in mobile health apps used by frontline healthcare workers in LMICs. It merges three complementary approaches—ECDF-based engagement indicators, a unified engagement score, and survival analysis using CSF and LTRC forests—to characterize individual and population-level engagement, with a focus on the Safe Delivery App in India and Ethiopia. The study demonstrates how churn can be detected earlier via ECDF methods, how a single engagement score can summarize multifaceted activity, and how time-to-churn predictions can be made despite censoring, offering practical pathways to tailor interventions for health workers. The framework holds potential to improve training and support through adaptive content and reminders, ultimately aiming to enhance patient care and health outcomes.

Abstract

Mobile health apps are revolutionizing the healthcare ecosystem by improving communication, efficiency, and quality of service. In low- and middle-income countries, they also play a unique role as a source of information about health outcomes and behaviors of patients and healthcare workers, while providing a suitable channel to deliver both personalized and collective policy interventions. We propose a framework to study user engagement with mobile health, focusing on healthcare workers and digital health apps designed to support them in resource-poor settings. The behavioral logs produced by these apps can be transformed into daily time series characterizing each user's activity. We use probabilistic and survival analysis to build multiple personalized measures of meaningful engagement, which could serve to tailor content and digital interventions suiting each health worker's specific needs. Special attention is given to the problem of detecting churn, understood as a marker of complete disengagement. We discuss the application of our methods to the Indian and Ethiopian users of the Safe Delivery App, a capacity-building tool for skilled birth attendants. This work represents an important step towards a full characterization of user engagement in mobile health applications, which can significantly enhance the abilities of health workers and, ultimately, save lives.
Paper Structure (18 sections, 2 equations, 9 figures, 4 tables)

This paper contains 18 sections, 2 equations, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Our contribution
  4. Dataset and Methodology
  5. Dataset
  6. Returning churners and missed metrics
  7. Empirical cumulative distribution function
  8. Engagement scores
  9. Survival analysis
  10. Conditional Survival Forests (CSF)
  11. Left-Truncated and Right-Censored (LTRC) Forests
  12. Results and discussion
  13. Returning churners and missed metrics
  14. ECDF indicator
  15. Engagement score
  16. ...and 3 more sections

Figures (9)

  • Figure 1: Kaplan--Meier estimates for India and Ethiopia. Shaded areas show 95% confidence intervals.
  • Figure 2: Fraction of returning churners ( top) and missed progression ( bottom) for India ( green) and Ethiopia ( pink), when we define churn as $k$ consecutive days without logins.
  • Figure 3: Histogram of days between logins for all users in India and Ethiopia, considering all historical data.
  • Figure 4: Endogenous ECDF distribution of days between logins for users 1 ( blue) and 2 ( red). The 0.9 probability horizontal line marks the churn alert threshold.
  • Figure 5: Distribution (kernel density estimation) of ECDF-snp values for August 1, 2021, across India and Ethiopia.
  • ...and 4 more figures