"My toxic trait is thinking I'll remember this": gaps in the learner experience of video tutorials for feature-rich software

TL;DR

This study investigates gaps in learner experiences when using video tutorials for feature-rich software, using Microsoft Excel as a case study. It analyzes 360 viewer comments from 90 tutorials across YouTube, TikTok, and Instagram to derive a taxonomy of 13 gaps spanning creator-driven, learner-driven, and app-driven factors. It triangulates comment analysis with contextual interviews of eight tutorial creators and introduces two design prototypes aimed at helping learners and creators mind these gaps, followed by a design probe. The findings reveal how tutorial content, platform constraints, and learner contexts interact to create barriers, and demonstrate that in-application interactive tutorials and creator-facing gap-detection tools can mitigate many barriers. The work offers design implications and points to AI-enabled approaches, including large language models, to support adaptive, version-aware, and multilingual learning resources for feature-rich software.

Abstract

Video tutorials are a popular medium for informal and formal learning. However, when learners attempt to view and follow along with these tutorials, they encounter what we call gaps, that is, issues that can prevent learning. We examine the gaps encountered by users of video tutorials for feature-rich software, such as spreadsheets. We develop a theory and taxonomy of such gaps, identifying how they act as barriers to learning, by collecting and analyzing 360 viewer comments from 90 Microsoft Excel video tutorials published by 43 creators across YouTube, TikTok, and Instagram. We conducted contextual interviews with 8 highly influential tutorial creators to investigate the gaps their viewers experience and how they address them. Further, we obtain insights into their creative process and frustrations when creating video tutorials. Finally, we present creators with two designs that aim to address gaps identified in the comment analysis for feedback and alternative design ideas.

Figures (6)

• Figure 1: Example of Excel video tutorials on YouTube (left) and TikTok (right): we find gaps that occur when learners use such videos.
• Figure 2: Diagram of the work presented in this paper. Study 1 is a social media comment analysis to discover categories of gaps that viewers face when using video tutorials (Sections \ref{['sec:commentmethods']} and \ref{['sec:resultgaps']}). Based on Study 1, we created two designs that attempted to address many of the gaps we had found (Section \ref{['sec:dprobemethods']}). We also created an interview protocol to gain insights from content creators (Section \ref{['sec:interviewmethods']}). In Study 2, we interviewed content creators about these gaps, their creation workflows, and their frustrations (Section \ref{['sec:results']}). Finally, we presented our designs to creators to obtain design implications for effective video tutorials (Section \ref{['sec:resultdprobe']}).
• Figure 3: A frame of Design 1, which shows interactive video tutorials within Excel. Shows a user encountering a #VALUE error. In response, the system queries the web for the issue and recommends relevant tutorials. As the tutorial plays, the user follows along with mirrored annotations within the application and completes each tutorial step (Figure \ref{['fig:d1steps']}).
• Figure 4: Example of the step-by-step nature of Design 1, where the video tutorial pauses and waits for the user to complete a step (from step 1 to step 2).
• Figure 5: Design 2, which shows gaps detected in a content creator's video tutorial. As the creator works within a worksheet, a list of potential gaps is created that the user can view. Beyond the gaps detected, recommendations for addressing these gaps are also provided to improve video tutorial effectiveness.