Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Applying Think-Aloud in ICTD: A Case Study of a Chatbot Use by Teachers in Rural Côte d'Ivoire

Vikram Kamath Cannanure, Sharon Wolf, Kaja Jasińska, Timothy X Brown, Amy Ogan

TL;DR

This paper investigates the applicability of think-aloud usability testing in ICTD contexts by conducting 20 sessions with rural teachers in Côte d'Ivoire two weeks after chatbot deployment. It uses a USSD-based chatbot and thematic analysis of think-aloud transcripts plus follow-up interviews to identify usability issues and learning curves. Key results show broad navigability but feature-specific challenges, with experts benefiting from menu-based navigation and Free Basics while novices face reading and input difficulties, emphasizing a learning curve and training effects. The work advances culturally sensitive adaptation of think-aloud methods for the Global South and provides practical guidance for designing, training, and evaluating classroom-focused chatbot tools in low-resource settings.

Abstract

Think-alouds are a common HCI usability method where participants verbalize their thoughts while using interfaces. However, their utility in cross-cultural settings, particularly in the Global South, is unclear, where cultural differences impact user interactions. This paper investigates the usability challenges teachers in rural Côte d'Ivoire faced when using a chatbot designed to support an educational program. We conducted think-aloud sessions with 20 teachers two weeks after a chatbot deployment, analyzing their navigation, errors, and time spent on tasks. We discuss our approach and findings that helped us identify usability issues and challenging features for improving the chatbot designs. Our note summarizes our reflections on using think-aloud and contributes to discussions on its culturally sensitive adaptation in the Global South.

Applying Think-Aloud in ICTD: A Case Study of a Chatbot Use by Teachers in Rural Côte d'Ivoire

TL;DR

This paper investigates the applicability of think-aloud usability testing in ICTD contexts by conducting 20 sessions with rural teachers in Côte d'Ivoire two weeks after chatbot deployment. It uses a USSD-based chatbot and thematic analysis of think-aloud transcripts plus follow-up interviews to identify usability issues and learning curves. Key results show broad navigability but feature-specific challenges, with experts benefiting from menu-based navigation and Free Basics while novices face reading and input difficulties, emphasizing a learning curve and training effects. The work advances culturally sensitive adaptation of think-aloud methods for the Global South and provides practical guidance for designing, training, and evaluating classroom-focused chatbot tools in low-resource settings.

Abstract

Think-alouds are a common HCI usability method where participants verbalize their thoughts while using interfaces. However, their utility in cross-cultural settings, particularly in the Global South, is unclear, where cultural differences impact user interactions. This paper investigates the usability challenges teachers in rural Côte d'Ivoire faced when using a chatbot designed to support an educational program. We conducted think-aloud sessions with 20 teachers two weeks after a chatbot deployment, analyzing their navigation, errors, and time spent on tasks. We discuss our approach and findings that helped us identify usability issues and challenging features for improving the chatbot designs. Our note summarizes our reflections on using think-aloud and contributes to discussions on its culturally sensitive adaptation in the Global South.
Paper Structure (5 sections, 2 figures, 1 table)

This paper contains 5 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Findings
  4. Discussion
  5. System Design

Figures (2)

  • Figure 1: Think-aloud session with a teacher, conducted two weeks after training. The teachers were asked to use the chatbot features as the interviewer recorded their attempts. They were given three attempts per feature, and the entire session was recorded on video.
  • Figure 2: Flowchart of system features used in the study