TouchWalker: Real-Time Avatar Locomotion from Touchscreen Finger Walking

TL;DR

TouchWalker addresses real-time full-body avatar locomotion on touchscreens by mapping two-finger finger-walking input to per-frame motion synthesis through TouchWalker-MotionNet, a MoE-GRU-based neural generator, coupled with TouchWalker-UI for avatar-relative foot control. The MotionNet blends TransNet and PoseNet to produce coherent, temporally-contextual motions, guided by a multi-term loss including a dedicated foot-alignment component. In a user study against a virtual joystick baseline, TouchWalker enhances embodiment, enjoyment, and immersion, with clear strengths in precise foot placement and embodied rhythm, though it faces challenges in fast-paced, spatially constrained tasks. The work demonstrates a practical, tactile approach to expressive mobile-avatar control and shifts finger-walking from symbolic cues toward continuous, frame-by-frame motion synthesis for immersive experiences.

Abstract

We present TouchWalker, a real-time system for controlling full-body avatar locomotion using finger-walking gestures on a touchscreen. The system comprises two main components: TouchWalker-MotionNet, a neural motion generator that synthesizes full-body avatar motion on a per-frame basis from temporally sparse two-finger input, and TouchWalker-UI, a compact touch interface that interprets user touch input to avatar-relative foot positions. Unlike prior systems that rely on symbolic gesture triggers or predefined motion sequences, TouchWalker uses its neural component to generate continuous, context-aware full-body motion on a per-frame basis-including airborne phases such as running, even without input during mid-air steps-enabling more expressive and immediate interaction. To ensure accurate alignment between finger contacts and avatar motion, it employs a MoE-GRU architecture with a dedicated foot-alignment loss. We evaluate TouchWalker in a user study comparing it to a virtual joystick baseline with predefined motion across diverse locomotion tasks. Results show that TouchWalker improves users' sense of embodiment, enjoyment, and immersion.

Figures (9)

• Figure 1: Overview of TouchWalker-MotionNet. The model generates full-body avatar motion in real time from touchscreen input, incorporating foot contact positions, facing direction, and state history.
• Figure 2: (a) TouchWalker-UI. The touch region (bottom-right square) displays a top-down rendering of the ground beneath the avatar. (b) The touched position (yellow circle) is interpreted relative to the avatar’s horizontal position and orientation on the ground. The red and blue dotted lines indicate the avatar’s forward and lateral directions in both the 3D virtual space and the touch region. All visual markers are added to the figure for illustration only.
• Figure 3: Scenes from the user study tasks. T1–S1 to T1–S5 denote the five stages of Task 1 (Multi-Stage Navigation); T2 denotes Task 2 (Stepping Stones).
• Figure 4: Two control methods compared in the user study.
• Figure 5: Tutorial stages used in the user study. (a) Movement Direction Control: Participants navigated toward sequential green goal spheres using only movement input, with facing direction fixed. An arrow below the avatar indicated the off-screen goal direction. (b) Movement Speed Control: Participants adjusted movement speed to reach a forward goal while being pushed back, learning to modulate speed without facing control. (c) Facing Direction Control: Participants reached multiple goals without on-screen arrows, practicing both facing and movement control to locate and approach targets.