Development of the Lifelike Head Unit for a Humanoid Cybernetic Avatar `Yui' and Its Operation Interface
Mizuki Nakajima, Kaoruko Shinkawa, Yoshihiro Nakata
TL;DR
The paper tackles improving avatar-mediated remote communication by developing a lifelike head unit, Yui, with 28 facial actuation points and integrated gaze, expression, and mouth-control capabilities, operated via an immersive HMD-based interface. It presents a ROS2-based control framework, a wire-driven mechanical design with a differential neck linkage, and stereo vision and audio to convey depth and spatial cues, enabling richer presence for both operator and interlocutor. Key contributions include a 28-point facial actuation scheme mapped to Action Units, a perception-capable head unit with eye and ear sensors, and an architecture that synchronizes operator expressions with the avatar while presenting CA perception through stereo video and audio. The findings suggest that the system enhances non-verbal communication in remote interactions and lays groundwork for broader avatar-based telepresence with future plans for arms and full-body mobility.
Abstract
In the context of avatar-mediated communication, it is crucial for the face-to-face interlocutor to sense the operator's presence and emotions via the avatar. Although androids resembling humans have been developed to convey presence through appearance and movement, few studies have prioritized deepening the communication experience for both operator and interlocutor using android robot as an avatar. Addressing this gap, we introduce the ``Cybernetic Avatar `Yui','' featuring a human-like head unit with 28 degrees of freedom, capable of expressing gaze, facial emotions, and speech-related mouth movements. Through an eye-tracking unit in a Head-Mounted Display (HMD) and degrees of freedom on both eyes of Yui, operators can control the avatar's gaze naturally. Additionally, microphones embedded in Yui's ears allow operators to hear surrounding sounds in three dimensions, enabling them to discern the direction of calls based solely on auditory information. An HMD's face-tracking unit synchronizes the avatar's facial movements with those of the operator. This immersive interface, coupled with Yui's human-like appearance, enables real-time emotion transmission and communication, enhancing the sense of presence for both parties. Our experiments demonstrate Yui's facial expression capabilities, and validate the system's efficacy through teleoperation trials, suggesting potential advancements in avatar technology.