Learning Hanzi Character Through VR-Based Mortise-Tenon

TL;DR

This work tackles the challenge of Hanzi literacy by introducing HVRMT, a VR-based system that reframes Hanzi components as interactive mortise-tenon modules to provide embodied, culturally grounded learning. It combines 6DoF spatial tracking, LLM-driven morphological analysis, and tactile haptics to map strokes to structural radicals and guide learners through character assembly in a collaborative VR space. The approach contributes by integrating traditional craftsmanship with digital literacy education and enabling multi-user interaction to enhance engagement and memory retention. Practical impact includes richer cultural heritage transmission in digital ecosystems and a scalable path to expand character libraries and mortise-tenon variants for broader literacy training.

Abstract

This paper introduces a novel VR-based system that redefines the acquisition of Hanzi character literacy by integrating traditional mortise-tenon joinery principles (HVRMT).Addressing the challenge of abstract character memorization in digital learning,our system deconstructs Hanzi components into interactive "structural radicals"akin to wooden joint modules.Leveraging PICO's 6DoF spatial tracking and LLM's morphological analysis,learners assemble stroke sequences with haptic feedback simulating wood-to-wood friction.Our system also supports multiplayer online experiences, enhancing engagement and memory retention while preserving intangible cultural heritage. This innovative approach not only enhances engagement and memory retention but also reconstructs the craft wisdom embedded in Chinese writing systems, offering new pathways for preserving intangible cultural heritage in digital ecosystems.For the demo,please refer to this link{https://youtu.be/oUwfFTRpFyo}.

Figures (4)

• Figure 1: Overview of our HVRMT application.
• Figure 2: Two users scene design.
• Figure 3: System area design diagram.
• Figure 4: (a) Recipe table created based on the reusability of parts, (b) Results of user studies.