The Impact of 2D and 3D Gamified VR on Learning American Sign Language

TL;DR

This study investigates whether 2D or 3D gamified VR environments better support learning American Sign Language (ASL). It implements a Unity-based VR setup with a dictionary, quiz, and two versions of a Whack-a-Mole game (2D and 3D) to teach ASL digits 0–9, using Mediapipe for hand tracking and a three-stage Learn–Practice–Assess protocol. Usability was measured with the System Usability Scale, while user experience followed Schrepp's six scales, and learning performance was tracked via game scores. Findings indicate that 3D environments enhance attractiveness, efficiency, novelty, perspicuity, and stimulation, contributing to a more engaging experience, though dependability remains an area for improvement and 2D still offers easier learnability; a strong learning effect across attempts was observed, with a small detriment when participants swapped between environments. The work informs design choices for VR-based ASL education, highlighting the trade-offs between immersion and ease of use and suggesting pathways for future, larger-scale studies to validate and extend these results.

Abstract

Sign language has been extensively studied as a means of facilitating effective communication between hearing individuals and the deaf community. With the continuous advancements in virtual reality (VR) and gamification technologies, an increasing number of studies have begun to explore the application of these emerging technologies in sign language learning. This paper describes a user study that compares the impact of 2D and 3D games on the user experience in ASL learning. Empirical evidence gathered through questionnaires supports the positive impact of 3D game environments on user engagement and overall experience, particularly in relation to attractiveness, usability, and efficiency. Moreover, initial findings demonstrate a similar behaviour of 2D and 3D games in terms of enhancing user experience. Finally, the study identifies areas where improvements can be made to enhance the dependability and clarity of 3D game environments. These findings contribute to the understanding of how game-based approaches, and specifically the utilisation of 3D environments, can positively influence the learning experience of ASL.

Figures (5)

• Figure 1: The implemented 2D versus 3D gamified virtual environment for ASL learning.
• Figure 2: Quiz module for learning ASL
• Figure 3: 2D Whack-a-Mole Game (Left); 3D Whack-a-Mole Game(Right)
• Figure 4: Six scales of user experience
• Figure 5: Each item of six scales of user experience.