Does the Doer Effect Exist Beyond WEIRD Populations? Toward Analytics in Radio and Phone-Based Learning

TL;DR

The paper investigates whether the Doer Effect generalizes to non-WEIRD populations by analyzing Ugandan learners using basic radio and mobile-phone delivery (N=234). It employs linear regression with $z$-scored predictors to relate final exam performance to active practice (doing) and passive listening, including interactions with prior education and listening. Key findings show a robust Doer Effect in this setting ($1$ SD of doing → ~0.37–0.42 SD exam gain) with moderation by education level and prior listening, suggesting greater benefits of practice for learners with lower prior knowledge and a synergistic role for listening. The work demonstrates generalizability to low-tech, hard-to-reach populations and informs design considerations for contextually relevant active/passive learning opportunities in remote STEM education.

Abstract

The Doer Effect states that completing more active learning activities, like practice questions, is more strongly related to positive learning outcomes than passive learning activities, like reading, watching, or listening to course materials. Although broad, most evidence has emerged from practice with tutoring systems in Western, Industrialized, Rich, Educated, and Democratic (WEIRD) populations in North America and Europe. Does the Doer Effect generalize beyond WEIRD populations, where learners may practice in remote locales through different technologies? Through learning analytics, we provide evidence from N = 234 Ugandan students answering multiple-choice questions via phones and listening to lectures via community radio. Our findings support the hypothesis that active learning is more associated with learning outcomes than passive learning. We find this relationship is weaker for learners with higher prior educational attainment. Our findings motivate further study of the Doer Effect in diverse populations. We offer considerations for future research in designing and evaluating contextually relevant active and passive learning opportunities including leveraging familiar technology, increasing the number of practice opportunities, and aligning multiple data sources.

Figures (2)

• Figure 1: Instruction, Assessment, and Data Collection. A student may listen to radio instruction or practice through multiple-choice assessments on a phone. A database logs student responses via USSD messages.
• Figure 2: Changes in exam score across percentage of practice questions done out of the total available