Dynamic Object Catching with Quadruped Robot Front Legs

TL;DR

The paper addresses dynamic object catching with a standing quadruped by elevating onto its rear legs to expose the front limbs for interception. It presents an end-to-end framework that combines onboard vision (YOLOv8), gravity-informed trajectory prediction via ordinary least squares, and a Cartesian PD-based catching controller, with three strategies for selecting catching positions: plane intersection, closest point to the front-legs center, and a GMM of reachable catches. The GMM-based catching position yields the best performance, achieving up to 80% success in standard throws and robust results in harder cases, illustrating effective integration of perception, prediction, and manipulation on a mobile robot. This work demonstrates a practical approach to loco-manipulation using existing front legs for catching, enabling agile, autonomous interactions with dynamic objects in real time.

Abstract

This paper presents a framework for dynamic object catching using a quadruped robot's front legs while it stands on its rear legs. The system integrates computer vision, trajectory prediction, and leg control to enable the quadruped to visually detect, track, and successfully catch a thrown object using an onboard camera. Leveraging a fine-tuned YOLOv8 model for object detection and a regression-based trajectory prediction module, the quadruped adapts its front leg positions iteratively to anticipate and intercept the object. The catching maneuver involves identifying the optimal catching position, controlling the front legs with Cartesian PD control, and closing the legs together at the right moment. We propose and validate three different methods for selecting the optimal catching position: 1) intersecting the predicted trajectory with a vertical plane, 2) selecting the point on the predicted trajectory with the minimal distance to the center of the robot's legs in their nominal position, and 3) selecting the point on the predicted trajectory with the highest likelihood on a Gaussian Mixture Model (GMM) modelling the robot's reachable space. Experimental results demonstrate robust catching capabilities across various scenarios, with the GMM method achieving the best performance, leading to an 80% catching success rate. A video demonstration of the system in action can be found at https://youtu.be/sm7RdxRfIYg .

Figures (6)

• Figure 1: The Unitree Go1 quadruped robot elevated on its rear legs, detecting a thrown object, predicting a suitable catching position, and successfully catching it with its front legs.
• Figure 2: Control diagram for catching objects with a quadruped robot. (A) the quadruped elevates onto its rear legs with an optimized trajectory, leaving its front legs available to catch an object. The user subsequently throws an object. (B) and (C): an onboard camera is used to detect the object and extract its location, the system iteratively predicts a suitable catching location through successive frames while the object is in the air. (D) as the object reaches the predicted catching location, the quadruped controls its front legs to catch the object.
• Figure 3: Example trajectory prediction. The discrete blue points represent observed object positions, the red curve shows the predicted trajectory using the first 3 observed positions, and the green curve shows the predicted trajectory using all 6 observed positions.
• Figure 4: Illustrations of the different catching position identification methods.
• Figure 5: GMM fitted over 100 catching demonstrations. The GMM models the quadruped's reachable space and provides likelihood information for catching positions.