Gazebo Plants: Simulating Plant-Robot Interaction with Cosserat Rods

TL;DR

This work addresses the lack of dynamic, non-rigid plant models in robotics simulation by introducing a Gazebo plugin based on Cosserat rods and Position-Based Dynamics to model plant motion and fruit detachment during robot harvesting. It integrates with Gazebo (via a World plugin) to build plant graphs from cylinder–sphere representations, calibrates interactions against real demos, and demonstrates two harvesting strategies (stretching and bending) with realistic responses. The results show close qualitative agreement with real-world harvesting, validating the approach for synthetic data generation, algorithm development, and verification of agricultural robots while outlining limitations such as gravity sag and the need to tune the initial Darboux vector. The plugin enables training and evaluation of harvesting policies in a safe, scalable virtual environment, with potential to reduce labor costs and improve agricultural productivity through more capable autonomous systems.

Abstract

Robotic harvesting has the potential to positively impact agricultural productivity, reduce costs, improve food quality, enhance sustainability, and to address labor shortage. In the rapidly advancing field of agricultural robotics, the necessity of training robots in a virtual environment has become essential. Generating training data to automatize the underlying computer vision tasks such as image segmentation, object detection and classification, also heavily relies on such virtual environments as synthetic data is often required to overcome the shortage and lack of variety of real data sets. However, physics engines commonly employed within the robotics community, such as ODE, Simbody, Bullet, and DART, primarily support motion and collision interaction of rigid bodies. This inherent limitation hinders experimentation and progress in handling non-rigid objects such as plants and crops. In this contribution, we present a plugin for the Gazebo simulation platform based on Cosserat rods to model plant motion. It enables the simulation of plants and their interaction with the environment. We demonstrate that, using our plugin, users can conduct harvesting simulations in Gazebo by simulating a robotic arm picking fruits and achieve results comparable to real-world experiments.

Figures (10)

• Figure 1: Visual comparison of simulation and real-world scenario: UFactory Lite 6 robotic arm picking cherry tomato using our custom gripper (left) and our corresponding simulation in the virtual environment (right).
• Figure 2: During initialization, Gazebo loads the world file, and within the parameters of our plugin, users need to specify both the plant file and the name of the robot intended to interact with the plant. Subsequently, the plugin proceeds to generate the plant as a combination of cylinders and spheres within the Gazebo environment, retrieving the robot's geometric data from Gazebo. During each physics time step, the following sequence of actions unfolds: Firstly, the robot's pose is updated, then it interaction with the plant is computed, and lastly, the pose of each individual component of the plant is reported back accordingly. Users can manipulate the robot by interfacing with Gazebo through widely used libraries such as ROS and MoveIt.
• Figure 3: Illustration of the plan discretization in which $\{\mathbf{d}_{k}\}$ denotes the local coordinate system for the Cosserat rod.
• Figure 4: (a) portrays a plant through the use of multiple curves. Through the process of sampling along these curves, connecting the branches, and introducing leaves, we can transform it into (b), which represents a plant through a collection of particles linked by a graph. (c) depicts the plant, as represented by (b), in the physical world. (d) illustrates the action of harvesting a cluster of fruits from the plant.
• Figure 5: Plant behaviors with different $\sigma_{\text{distance}}=0.02, 0.035, 0.05~\text{m}$. For all three results, $\sigma_{\text{stiffness}}=2\times 10^{4}$ and $\rho=300~\text{kg}/\text{m}^{3}$.