Learning by Watching: Physical Imitation of Manipulation Skills from Human Videos

TL;DR

Learning by Watching (LbW) tackles robotic manipulation by learning from a single human video. It combines unsupervised human-to-robot translation (via MUNIT) with unsupervised keypoint detection (via Transporter) to produce semantically meaningful representations that define rewards and drive RL via SAC. The method demonstrates favorable performance across five tasks in two simulators, reducing translation artifacts and outperforming or matching state-of-the-art baselines. By leveraging keypoint-based state representations for visuomotor imitation, LbW offers a scalable approach to learning manipulation skills from human demonstrations.

Abstract

Learning from visual data opens the potential to accrue a large range of manipulation behaviors by leveraging human demonstrations without specifying each of them mathematically, but rather through natural task specification. In this paper, we present Learning by Watching (LbW), an algorithmic framework for policy learning through imitation from a single video specifying the task. The key insights of our method are two-fold. First, since the human arms may not have the same morphology as robot arms, our framework learns unsupervised human to robot translation to overcome the morphology mismatch issue. Second, to capture the details in salient regions that are crucial for learning state representations, our model performs unsupervised keypoint detection on the translated robot videos. The detected keypoints form a structured representation that contains semantically meaningful information and can be used directly for computing reward and policy learning. We evaluate the effectiveness of our LbW framework on five robot manipulation tasks, including reaching, pushing, sliding, coffee making, and drawer closing. Extensive experimental evaluations demonstrate that our method performs favorably against the state-of-the-art approaches.

Figures (5)

• Figure 1: LbW. Given a single human video, our LbW framework learns human to robot translation followed by unsupervised keypoint detection. The resulting keypoint-based representations are semantically meaningful and can be used to guide the robot to learn manipulation skills through physical imitation.
• Figure 2: Overview of the proposed LbW. Our LbW framework is composed of three main components: an image-to-image translation network $T$, a keypoint detector $\Psi$, and a policy network $\pi$. The image-to-image translation network translates the input human demonstration video frame by frame to generate a robot demonstration video. Next, the keypoint detector takes the generated robot demonstration video as input and extracts the keypoint-based representation for each frame to form a keypoints trajectory. At each time step, the keypoint detector also extracts the keypoint-based representation for the current observation. The reward for physical imitation is defined by a distance metric $d$ that measures the distance between the keypoint-based representation of the current observation and those in the keypoints trajectory. Finally, the keypoint-based representation of the current observation is passed to the policy network to predict an action that is used to interact with the environment.
• Figure 3: Overview of the perception module. Our perception module is composed of a MUNIT network (left) and a Transporter model (right). Given a human video frame $x$ and a robot video frame $y$, the MUNIT model first extracts the content code of the human video frame and the style code of the robot video frame. The MUNIT model then generates the translated robot video frame $v$ by combining the extracted content code and style code. Next, the Transporter model extracts the features and detects the keypoints for both the translated robot video frame $v$ and the input robot video frame $y$ and reconstructs the translated robot video frame $\hat{v}$ by transporting features at the detected keypoint locations. Note that the input robot video frame $y$ is from a robot video generated by using a random policy.
• Figure 4: Task overview. We present the sample task scenes and one sample human video frame for the pushing, sliding, drawer closing, and coffee making tasks, respectively. Our human videos can be collected in an environment with a plain background (i.e., the left three columns) or with a noisy background (i.e., the rightmost column).
• Figure 5: Visual results and comparisons on the pushing task. Given a human video as input in the first row, we present the translated images of CycleGAN cyclegan in the second row. In the third row, we visualize our translated images and the detected keypoints produced by the perception module. Our perception module accurately detects the robot arm pose and the location of the interacting object.