Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition

TL;DR

A novel system for joint learning between human operators and robots, that enables human operators to share control of a robot end-effector with a learned assistive agent, simplifies the data collection process, and facilitates simultaneous human demonstration collection and robot manipulation training is introduced.

Abstract

Employing a teleoperation system for gathering demonstrations offers the potential for more efficient learning of robot manipulation. However, teleoperating a robot arm equipped with a dexterous hand or gripper, via a teleoperation system presents inherent challenges due to the task's high dimensionality, complexity of motion, and differences between physiological structures. In this study, we introduce a novel system for joint learning between human operators and robots, that enables human operators to share control of a robot end-effector with a learned assistive agent, simplifies the data collection process, and facilitates simultaneous human demonstration collection and robot manipulation training. As data accumulates, the assistive agent gradually learns. Consequently, less human effort and attention are required, enhancing the efficiency of the data collection process. It also allows the human operator to adjust the control ratio to achieve a trade-off between manual and automated control. We conducted experiments in both simulated environments and physical real-world settings. Through user studies and quantitative evaluations, it is evident that the proposed system could enhance data collection efficiency and reduce the need for human adaptation while ensuring the collected data is of sufficient quality for downstream tasks. \textit{For more details, please refer to our webpage https://norweig1an.github.io/HAJL.github.io/.

Figures (6)

• Figure 1: Human-agent joint learning overview. Traditional frameworks typically separate human and agent training, requiring operators first to learn the task environment before data collection. This often leads to inefficiencies due to delayed and insufficient data gathering. In our framework, we integrate human and agent training from the start in a joint learning model. This enables simultaneous development and adapts the agents to human operation more effectively, enhancing overall efficiency and promoting better collaboration between humans and machines allowing for human effortless adaptation data collection.
• Figure 2: Simulation tasks overview. Here are six task settings and their task flow for Pick-and-Place (left), Articulated-Manipulation (middle), Gripper-Push (upper-right) and Dexterous-Tool-Use (bottom-right).
• Figure 3: Diffusion based shared control. To achieve shared control between the human and agent, we blend the action from the human operator $a^h$ using the forward and reverse process. The parameter $\gamma$ governs the control ratio, where a lower $\gamma$ results in the action better aligning with the human operator's intention. In contrast, a higher $\gamma$ allows the learned agent to exert more influence over the blended action.
• Figure 4: Shared control process overview. The white one is the hand controlled purely by the human operator, while the cyan one is under shared control between the human and the assistive agent.
• Figure 5: Agent performance over time. The x-axis represents the control ratio $\gamma$ and the y-axis represents the success rate. We train a simulated operator to evaluate our system, it shows that even with limited data, the learned assist agent can improve the success rate of data collection to improve the efficiency. With the data accumulated, the performance of the learned agent keeps rising. Moreover, the learned agent could be transitioned to a full autonomy agent ($\gamma=1.0$).