Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AfforDance: Personalized AR Dance Learning System with Visual Affordance

Hyunyoung Han, Jongwon Jang, Kitaeg Shim, Sang Ho Yoon

TL;DR

This work tackles accessible, personalized dance learning by shifting from handheld video-based instruction and immersive HMDs toward AR-based guidance with visual affordances. It introduces AfforDance, an AR system that converts user-selected dance videos into learnable content via a three-component pipeline: learning content generation (audio embedding and 3D reference avatars), affordance generation, and an AR user interface integrated in Unity. The approach leverages an 8-count beat, WHAM-based 3D pose extraction, and real-time pose-aligned affordances to support self-paced learning, aiming to reduce fatigue and increase engagement. By enabling large-display AR overlays and interactive cues, the method offers a practical, user-centered path for XR-assisted dance education with potential for broader accessibility.

Abstract

We propose AfforDance, an augmented reality (AR)-based dance learning system that generates personalized learning content and enhances learning through visual affordances. Our system converts user-selected dance videos into interactive learning experiences by integrating 3D reference avatars, audio synchronization, and adaptive visual cues that guide movement execution. This work contributes to personalized dance education by offering an adaptable, user-centered learning interface.

AfforDance: Personalized AR Dance Learning System with Visual Affordance

TL;DR

This work tackles accessible, personalized dance learning by shifting from handheld video-based instruction and immersive HMDs toward AR-based guidance with visual affordances. It introduces AfforDance, an AR system that converts user-selected dance videos into learnable content via a three-component pipeline: learning content generation (audio embedding and 3D reference avatars), affordance generation, and an AR user interface integrated in Unity. The approach leverages an 8-count beat, WHAM-based 3D pose extraction, and real-time pose-aligned affordances to support self-paced learning, aiming to reduce fatigue and increase engagement. By enabling large-display AR overlays and interactive cues, the method offers a practical, user-centered path for XR-assisted dance education with potential for broader accessibility.

Abstract

We propose AfforDance, an augmented reality (AR)-based dance learning system that generates personalized learning content and enhances learning through visual affordances. Our system converts user-selected dance videos into interactive learning experiences by integrating 3D reference avatars, audio synchronization, and adaptive visual cues that guide movement execution. This work contributes to personalized dance education by offering an adaptable, user-centered learning interface.
Paper Structure (13 sections, 3 figures, 1 table)

This paper contains 13 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Exploratory Study
  3. Study Design
  4. Findings and Design Goals
  5. Device Usage Disrupts Learning Flow
  6. Difficulty in Learning Dance Movements Independently
  7. Design Goals
  8. Affordance
  9. Learning Content Generation
  10. Affordance
  11. Design and Operation of Affordance
  12. Affordance Generation
  13. Future Direction for Dance Guidance in XR

Figures (3)

  • Figure 1: The learning content generation process consists of three stages: (1) Audio content generation, where an 8-count voice beat is added to the extracted audio; (2) Reference avatar generation, creating a 3D avatar from the dance video; and (3) Affordance generation, adjusting affordances to match the user's body size using webcam data.
  • Figure 2: The affordance can be presented as (a) only certain joint areas such as hands and legs, (b) certain joint areas and the rest of the body with transparency, or (c) all body parts without transparency.
  • Figure 3: Our user interface consists of four elements: (a) user appearance and affordances, (b) reference avatar, (c) learning progress bar, and (d) learning support features.