A Taxonomy of Self-Handover

TL;DR

This work introduces a dedicated taxonomy for self-handover, a common but understudied bimanual action, founded on over 12 hours of egocentric cooking video from 21 participants. It defines a two-axis framework (primary actor and task focus) within a 3x3 design and identifies five observed categories that describe how self-handover reorganizes subsequent actions. The authors provide a functional analysis of self-handover strategies, demonstrating their role in improving task efficiency, and validate a preliminary classification approach using a vision-language model (GPT-4o) on cooking data, achieving strong overall accuracy while noting hallucination risks. The findings advance understanding of human bimanual coordination and offer a practical basis for developing context-aware, adaptive self-handover behaviors in dual-arm robotics, with implications for automated analysis and robotic manipulation.

Abstract

Self-handover, transferring an object between one's own hands, is a common but understudied bimanual action. While it facilitates seamless transitions in complex tasks, the strategies underlying its execution remain largely unexplored. Here, we introduce the first systematic taxonomy of self-handover, derived from manual annotation of over 12 hours of cooking activity performed by 21 participants. Our analysis reveals that self-handover is not merely a passive transition, but a highly coordinated action involving anticipatory adjustments by both hands. As a step toward automated analysis of human manipulation, we further demonstrate the feasibility of classifying self-handover types using a state-of-the-art vision-language model. These findings offer fresh insights into bimanual coordination, underscoring the role of self-handover in enabling smooth task transitions-an ability essential for adaptive dual-arm robotics.

Figures (8)

• Figure 1: A functional taxonomy of self-handover behaviors. Self-handover---the transfer of an object between one’s own hands---takes on different forms depending on the goal of the handover and the roles of each hand. We identify six core patterns, structured by whether the handover targets an object or a hand, and whether one or both hands remain active afterward. Red cells indicate active self-handovers, where the object plays a central role in the next action. Blue marks passive transitions, where the object is temporarily released to free a hand. Full definitions of each category are provided in Section \ref{['taxonomy']}.
• Figure 2: An example of self-handover for adjusting grip. The numbers indicate the chronological order of frames.
• Figure 3: An example that can be interpreted as both "Freeing the Hand for a New Task" and "Simplifying the Subsequent Task." After handing over the daikon radish, the left hand turns on the faucet while the right hand holds and rinses the vegetable under running water. The numbers indicate the chronological order of frames.
• Figure 4: (a) Overview of the kitchen setup, consisting of the washing area (orange), cutting area (green), and heating area (orange). (b) Sample image obtained from the head-mounted GoPro used in the analysis.
• Figure 5: The prompt for detecting self-handover.