Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Vision-assisted Avocado Harvesting with Aerial Bimanual Manipulation

Zhichao Liu, Jingzong Zhou, Caio Mucchiani, Konstantinos Karydis

TL;DR

The paper tackles autonomous avocado harvesting in unstructured orchard canopies where ground-based harvesters struggle. It introduces a bimanual aerial robot with a fixer arm to stabilize the peduncle and a gripper arm to detach the fruit, guided by a vision-and-learning pipeline that detects avocados and estimates their 3D pose. Pose estimates feed a MoveIt-based planner to generate staged poses for both arms, enabling onboard, real-time autonomy. Experimental validation includes lab simulations, field tests with real avocados, and complete system demonstrations, showing feasibility and paving the way for field deployment.

Abstract

Robotic fruit harvesting holds potential in precision agriculture to improve harvesting efficiency. While ground mobile robots are mostly employed in fruit harvesting, certain crops, like avocado trees, cannot be harvested efficiently from the ground alone. This is because of unstructured ground and planting arrangement and high-to-reach fruits. In such cases, aerial robots integrated with manipulation capabilities can pave new ways in robotic harvesting. This paper outlines the design and implementation of a bimanual UAV that employs visual perception and learning to autonomously detect avocados, reach, and harvest them. The dual-arm system comprises a gripper and a fixer arm, to address a key challenge when harvesting avocados: once grasped, a rotational motion is the most efficient way to detach the avocado from the peduncle; however, the peduncle may store elastic energy preventing the avocado from being harvested. The fixer arm aims to stabilize the peduncle, allowing the gripper arm to harvest. The integrated visual perception process enables the detection of avocados and the determination of their pose; the latter is then used to determine target points for a bimanual manipulation planner. Several experiments are conducted to assess the efficacy of each component, and integrated experiments assess the effectiveness of the system.

Vision-assisted Avocado Harvesting with Aerial Bimanual Manipulation

TL;DR

The paper tackles autonomous avocado harvesting in unstructured orchard canopies where ground-based harvesters struggle. It introduces a bimanual aerial robot with a fixer arm to stabilize the peduncle and a gripper arm to detach the fruit, guided by a vision-and-learning pipeline that detects avocados and estimates their 3D pose. Pose estimates feed a MoveIt-based planner to generate staged poses for both arms, enabling onboard, real-time autonomy. Experimental validation includes lab simulations, field tests with real avocados, and complete system demonstrations, showing feasibility and paving the way for field deployment.

Abstract

Robotic fruit harvesting holds potential in precision agriculture to improve harvesting efficiency. While ground mobile robots are mostly employed in fruit harvesting, certain crops, like avocado trees, cannot be harvested efficiently from the ground alone. This is because of unstructured ground and planting arrangement and high-to-reach fruits. In such cases, aerial robots integrated with manipulation capabilities can pave new ways in robotic harvesting. This paper outlines the design and implementation of a bimanual UAV that employs visual perception and learning to autonomously detect avocados, reach, and harvest them. The dual-arm system comprises a gripper and a fixer arm, to address a key challenge when harvesting avocados: once grasped, a rotational motion is the most efficient way to detach the avocado from the peduncle; however, the peduncle may store elastic energy preventing the avocado from being harvested. The fixer arm aims to stabilize the peduncle, allowing the gripper arm to harvest. The integrated visual perception process enables the detection of avocados and the determination of their pose; the latter is then used to determine target points for a bimanual manipulation planner. Several experiments are conducted to assess the efficacy of each component, and integrated experiments assess the effectiveness of the system.
Paper Structure (17 sections, 4 equations, 15 figures, 6 tables)

This paper contains 17 sections, 4 equations, 15 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Design and Analysis of the Bimanual Aerial Robot
  3. Overall Design and Major Components
  4. Design and Kinematic Analysis of the Dual-arm System
  5. Gripper End-effector Design
  6. Fixer End-effector Design
  7. Forward Kinematics
  8. Workspace Analysis
  9. Learning-based Visual Perception and Manipulation Planning for Autonomous Operation
  10. Autonomous Bimanual Aerial Robot Avocado Harvesting
  11. Object Detection and Segmentation
  12. Pose Estimation
  13. Experimental Results and Discussion
  14. Preliminary Feasibility Experiment with the Dual-arm System
  15. Dual-arm System Testing in Field Experiments with Real Avocados
  16. ...and 2 more sections

Figures (15)

  • Figure 1: (a) The prototype bimanual aerial robot developed in this work. (b) Detailed view of key individual components.
  • Figure 2: Detailed Dual-arm Assembly and Associated Coordinate System Frames.
  • Figure 3: We employ three stages to harvest an avocado. (a) The arm brings the end-effector close to the avocado. (b) The base rotates to bring the fingers in contact with the avocado. (c) Once secured, the wrist rotates to apply a moment to the avocado to detach it from its peduncle and retrieve it.
  • Figure 4: We employ two stages to secure the avocado peduncle. (a) The arm brings the end-effector close to the peduncle. (b) The active part rotates to come in contact with the peduncle.
  • Figure 5: Workspace of the dual-arm system. (Figure best viewed in color.)
  • ...and 10 more figures