Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Zero-Shot Sim-to-Real Reinforcement Learning for Fruit Harvesting

Emlyn Williams, Athanasios Polydoros

TL;DR

The paper addresses robust autonomous harvesting in unstructured farming environments by building a sim-to-real pipeline that uses FruitGym, a domain-randomized Mujoco environment, and a DRM-based RL policy. A two-layer control stack combines a high-level policy (operating at 20 Hz) with a Cartesian impedance controller (1 kHz) and dual-camera perception to enable end-to-end manipulation. Key findings show that DRM can learn grabbing behavior in simulation and transfer to real hardware with reasonable performance, albeit with performance gaps under clutter and substantial computational demands. The work offers a practical, zero-shot transfer framework for agricultural robotics and outlines concrete paths for scaling to more crops and real-world validation.

Abstract

This paper presents a comprehensive sim-to-real pipeline for autonomous strawberry picking from dense clusters using a Franka Panda robot. Our approach leverages a custom Mujoco simulation environment that integrates domain randomization techniques. In this environment, a deep reinforcement learning agent is trained using the dormant ratio minimization algorithm. The proposed pipeline bridges low-level control with high-level perception and decision making, demonstrating promising performance in both simulation and in a real laboratory environment, laying the groundwork for successful transfer to real-world autonomous fruit harvesting.

Zero-Shot Sim-to-Real Reinforcement Learning for Fruit Harvesting

TL;DR

The paper addresses robust autonomous harvesting in unstructured farming environments by building a sim-to-real pipeline that uses FruitGym, a domain-randomized Mujoco environment, and a DRM-based RL policy. A two-layer control stack combines a high-level policy (operating at 20 Hz) with a Cartesian impedance controller (1 kHz) and dual-camera perception to enable end-to-end manipulation. Key findings show that DRM can learn grabbing behavior in simulation and transfer to real hardware with reasonable performance, albeit with performance gaps under clutter and substantial computational demands. The work offers a practical, zero-shot transfer framework for agricultural robotics and outlines concrete paths for scaling to more crops and real-world validation.

Abstract

This paper presents a comprehensive sim-to-real pipeline for autonomous strawberry picking from dense clusters using a Franka Panda robot. Our approach leverages a custom Mujoco simulation environment that integrates domain randomization techniques. In this environment, a deep reinforcement learning agent is trained using the dormant ratio minimization algorithm. The proposed pipeline bridges low-level control with high-level perception and decision making, demonstrating promising performance in both simulation and in a real laboratory environment, laying the groundwork for successful transfer to real-world autonomous fruit harvesting.
Paper Structure (14 sections, 7 equations, 7 figures)

This paper contains 14 sections, 7 equations, 7 figures.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Sim to Real transfer in Reinforcement Learning
  4. Reinforcement Learning for Agricultural Robotics
  5. Simulation Environment
  6. Low-Level Control via Impedance Controller
  7. Policy Learning
  8. Reward Function
  9. Real Robot setup
  10. Experiments
  11. Results and Discussion
  12. Limitations
  13. Future Work
  14. CONCLUSION

Figures (7)

  • Figure 1: System overview illustrating our sim-to-real pipeline for robotic fruit harvesting. In the top panel (Simulation), the RL agent is trained in the custom FruitGym MuJoCo environment, receiving visual and pose observations $s$, reward $r$ and outputting control actions $a$. A Cartesian impedance controller then applies these actions at high frequency in simulation. Once trained, the same policy is deployed on the real Franka Panda robot (bottom panel), where the robot’s vision and end-effector pose are fed back to the trained policy for closed-loop control.
  • Figure 2: FruitGym environments with domain randomization.
  • Figure 3: Examples of successful attempts in sim and real environments in a cluster containing 5 green strawberries and 1 red.
  • Figure 4: Evaluation cluster examples with varying number of green strawberries.
  • Figure 5: RL episode reward throughout training. Results are averaged over 3 seeds with a standard deviation shading of $\pm0.5$.
  • ...and 2 more figures