From Formulas to Figures: How Visual Elements Impact User Interactions in Educational Videos

TL;DR

The paper investigates how visual complexity in STEM educational videos shapes real-world user interactions, addressing a gap between laboratory findings and authentic platform data. It introduces a fine-grained visual-object taxonomy, defines visual complexity as frame-to-frame object changes, and employs a permutation-based testing framework with a modified Dynamic Time Warping alignment to link complexity changes to user actions such as pauses, rewinds, and dropouts. Results show that textual complexity significantly increases pausing ($p<0.05$, PES ≈ 0.64) and dropout/rewind tendencies, while purely visual content exhibits weaker effects, highlighting cognitive-load considerations for text-heavy segments. The work offers design guidance (complexity assessment tools, segment cues, and interactive elements) and provides open-source code at https://github.com/TIBHannover/from_formulas_to_figures to enable replication and extension.

Abstract

Educational videos have become increasingly relevant in today's learning environments. While prior research in laboratory studies has provided valuable insights, analyzing real-world interaction data can enhance our understanding of authentic user behavior. Previous studies have investigated technical aspects, such as the influence of cuts on pausing behavior, but the impact of visual complexity remains understudied. In this paper, we address this gap and propose a novel approach centered on visual complexity, defined as the number of visually distinguishable and meaningful elements in a video frame, such as mathematical equations, chemical formulas, or graphical representations. Our study introduces a fine-grained taxonomy of visual objects in educational videos, expanding on previous classifications. Applying this taxonomy to 25 videos from physics and chemistry, we examine the relationship between visual complexity and user behavior, including pauses, in-video navigation, and session dropouts. The results indicate that increased visual complexity, especially of textual elements, correlates with more frequent pauses, rewinds, and dropouts. The results offer a deeper understanding of how video design affects user behavior in real-world scenarios. Our work has implications for optimizing educational videos, particularly in STEM fields. We make our code publicly available (https://github.com/TIBHannover/from_formulas_to_figures).

Figures (3)

• Figure 1: Shows the visualization of the developed taxonomy for visual objects in chemistry, physics, and mathematics videos.
• Figure 2: Annotated example frame (see 4:46 at https://av.tib.eu/media/15698)
• Figure 3: Overview of our data preprocessing (Section \ref{['sec:preprocessing']}).