Motion Design Principles for Accessible Video-based Learning: Addressing Cognitive Challenges for Deaf and Hard of Hearing Learners

TL;DR

Deaf and Hard-of-Hearing learners face cognitive challenges in video-based learning due to the need to process captions and visual content simultaneously. The authors propose Motion-driven design guidelines and test them through a two-phase study that first identifies challenges in mainstream videos and then evaluates the perceived value of motion-driven edits with 16 DHH participants. Results show that improving visual-audio relevance, especially via D-Illustrate, and guiding visual attention significantly reduce cognitive demands and boost learning satisfaction, with other approaches yielding mixed effects. The work advances scalable design principles for inclusive video production and points to AI-assisted workflows to automate motion-driven accessibility interventions in educational video content.

Abstract

Deaf and Hard-of-Hearing (DHH) learners face unique challenges in video-based learning due to the complex interplay between visual and auditory information in videos. Traditional approaches to making video content accessible primarily focus on captioning, but these solutions often neglect the cognitive demands of processing both visual and textual information simultaneously. This paper introduces a set of \textit{Motion} design guidelines, aimed at mitigating these cognitive challenges and improving video learning experiences for DHH learners. Through a two-phase research, we identified five key challenges, including misaligned content and visual overload. We proposed five design principles accordingly. User study with 16 DHH participants showed that improving visual-audio relevance and guiding visual attention significantly enhances the learning experience by reducing physical demand, alleviating temporal pressure, and improving learning satisfaction. Our findings highlight the potential of Motion design to transform educational content for DHH learners, and we discuss implications for inclusive video learning tools.

Figures (10)

• Figure 1: Original vs Revised Video Pairs used in RQ2 Following Four Motion-Driven Design Ideas (challenge-suggestion pairs) Identified in RQ1.
• Figure 2: Study Procedure for RQ2. During onboarding, participants were introduced to all design ideas described in Table \ref{['tab:RQ1findings']}. Then, they watched 17 video pairs. Each pair of videos consists of an original (unedited) version and a revised (edited) version with one of the four design ideas applied. For each pair of videos, participants will watch both versions in a randomized order (either showing the original version first, or the revised version first, as demonstrated by two examples in the figure), and then complete a four-question survey developed from NASA Task Load Index for each version they watched (TLX questions). Then, they will answer three additional survey questions on learning cognitive demand scores (LCD questions) comparing the two versions. The same survey questions were used for all 17 video pairs. Participants were given at least one 5-minute break during video rating. Then, participants took place in a brief interview to discuss about their suggestions toward the design ideas they experienced.
• Figure 3: Survey results of using design ideas to support multimedia learning (LCD Questions), shown in a violin plot. Video clips for each design idea were aggregated. Post-hoc analysis on linear mixed models for each question suggested that for Focusing on Essential Information, there is significant difference between D-Illustrate and D-Slowdown. For Fostering Connection between Text and Image, there is significant difference between D-Illustrate and D-Declutter. The ** denotes significant differences between the two design ideas at p<.01.
• Figure A.1: Example Row from the Participant Canvas (One of Multiple Similar Entries)
• Figure A.2: Participant's challenge-suggestion: To enhance readability, shorten the bullet points, and display the remainder on the next clip if there are existing subtitles. This will prevent the screen from becoming too cluttered. This informed the theme D-Declutter. Note: The participant later explained to the researcher that putting the text in the middle of the screen and potentially removing the presenter would increase focus.