iCub3 Avatar System: Enabling Remote Fully-Immersive Embodiment of Humanoid Robots

TL;DR

This work tackles remote embodiment of humans in humanoid robots through telexistence by introducing iCub3 as a capable avatar platform and a generic, multi-modal avatar system. The authors combine locomotive, manipulation, speech, and facial expression control with rich sensory feedback (visual, auditory, haptic, weight, touch) and validate across remote demonstrations, live-stage engagement, and ANA Avatar XPrize scenarios. They deploy a modular, agile development approach leveraging lightweight operator wearables (iFeel, VR gear, trackers) and multiple locomotion interfaces (Virtualizer and iFeel Walking), achieving teleoperation over hundreds of kilometers with bipedal capabilities. The study yields practical design guidance on balancing transparency and operator effort, modularity and usability, and off-the-shelf versus in-house technologies, contributing a concrete blueprint for immersive humanoid telepresence with real-world applicability and competition relevance.

Abstract

We present an avatar system designed to facilitate the embodiment of humanoid robots by human operators, validated through iCub3, a humanoid developed at the Istituto Italiano di Tecnologia (IIT). More precisely, the contribution of the paper is twofold: first, we present the humanoid iCub3 as a robotic avatar which integrates the latest significant improvements after about fifteen years of development of the iCub series; second, we present a versatile avatar system enabling humans to embody humanoid robots encompassing aspects such as locomotion, manipulation, voice, and face expressions with comprehensive sensory feedback including visual, auditory, haptic, weight, and touch modalities. We validate the system by implementing several avatar architecture instances, each tailored to specific requirements. First, we evaluated the optimized architecture for verbal, non-verbal, and physical interactions with a remote recipient. This testing involved the operator in Genoa and the avatar in the Biennale di Venezia, Venice - about 290 Km away - thus allowing the operator to visit remotely the Italian art exhibition. Second, we evaluated the optimised architecture for recipient physical collaboration and public engagement on-stage, live, at the We Make Future show, a prominent world digital innovation festival. In this instance, the operator was situated in Genoa while the avatar operates in Rimini - about 300 Km away - interacting with a recipient who entrusted the avatar a payload to carry on stage before an audience of approximately 2000 spectators. Third, we present the architecture implemented by the iCub Team for the ANA Avatar XPrize competition.

Figures (13)

• Figure 1: iCub3 explores the Biennale di Venezia. Snapshots of the videobiennale_video demonstrating the remote teleoperation of iCub3 at the Italian Pavillion of the Biennale di Venezia. The operator navigates the remote venue via iCub3 (A, B). The operator controls the iCub3 eyelids in response to bright light (C). The operator remotely grasps a piece of tissue through iCub3 (D). The robot is touched on the arm (E). The robot skin, whose activation is represented in (F), triggers the body haptic feedback on the operator.
• Figure 2: Face expressions retargeting and iFeel walking. Examples of the retargeting interfaces. With the facial tracker (A), the operator can directly control the emotions displayed by the robot (B). The iFeel shoes (C). They measure the force and torque exchanged by the operator with the ground. When paired with a set of trackers, it is also possible to detect their position. An example of the intention mechanism used for the locomotion interface (D).
• Figure 3: iCub3 on the stage of the "We Make Future" Festival. iCub3 is on the stage of a tech fair, while being teleoperated from remotewmf_video. iCub3 interacts with the audience (A). iCub3 holding a box (B). Plot of the center of mass tracking of the robot while walking with a box (C). Plot of the vertical force measured on the robot arms while walking. A recipient hands the robot the box at around 10s. Then, the box is taken back by the recipient after the robot stopped walking (D). iCub3 interacts with the recipient (E).
• Figure 4: iCub3 at the XPrize semifinals. Pictures and plots of the iCub3 avatar system performance at the Xprize semifinals. iCub3 manipulating one puzzle piece (A). iCub3 checking the texture of the vase (B). iCub3 walking during the XPrize semifinals (C).CoM tracking during the walking task (D). The left and right hand Cartesian errors during the puzzle task (E) and (F). Magnified version of (E) to highlight the Cartesian tracking and lags (G). Normal force measured by the hands while lifting the vases and putting it back in place (H).
• Figure 5: iCub3 at the XPrize finals. Pictures of the iCub3 avatar system performance at the XPrize finals. The first three pictures have been shot by the authors while on the finals course. The last three pictures have been taken during tests on the lab prior to the finals. iCub3 walking on the course, themed on the exploration of another planet (A). iCub3 activating the switch using a plastic cylinder installed on the wrist (B). A portion of a video of the iCub3 finals trial, while hitting the door. The Operator's view is visible in the bottom right corner (C). iCub3 activating the drill (D). iCub3 holding the XPrize finals canisters (E). iCub3 making contact with a rough textured rock via the hand palm skin (F).