Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Strawberry Robotic Operation Interface: An Open-Source Device for Collecting Dexterous Manipulation Data in Robotic Strawberry Farming

Linsheng Hou, Wenwu Lu, Yanan Wang, Chen Peng, Zhenghao Fei

TL;DR

The paper presents the Strawberry Robotic Operation Interface (SROI), an open-source handheld platform designed to collect dexterous manipulation demonstrations for strawberry farming in field environments. It combines a modular cut-and-hold end effector, a low-cost electric gripper, and stereo cameras with a portable data-collection unit, enabling accurate trajectory and gripper-state recording. A stereo-IMU SLAM-based post-processing pipeline (using ORB-SLAM3) recovers spatial poses without pre-mapped scenes, and an AprilTag-based detector provides gripper state, facilitating data-ready trajectories for imitation-learning pipelines. The authors validate the system with field experiments across two camera setups, achieving consistent, high-precision trajectory estimates (low RMSE in translation) and releasing an open strawberry-picking dataset to accelerate research in dexterous manipulation in agriculture. Overall, SROI offers a scalable, field-friendly data collection solution that supports data-driven approaches to automate occluded strawberry picking and related tasks.

Abstract

The strawberry farming is labor-intensive, particularly in tasks requiring dexterous manipulation such as picking occluded strawberries. To address this challenge, we present the Strawberry Robotic Operation Interface (SROI), an open-source device designed for collecting dexterous manipulation data in robotic strawberry farming. The SROI features a handheld unit with a modular end effector, a stereo robotic camera, enabling the easy collection of demonstration data in field environments. A data post-processing pipeline is introduced to extract spatial trajectories and gripper states from the collected data. Additionally, we release an open-source dataset of strawberry picking demonstrations to facilitate research in dexterous robotic manipulation. The SROI represents a step toward automating complex strawberry farming tasks, reducing reliance on manual labor.

Strawberry Robotic Operation Interface: An Open-Source Device for Collecting Dexterous Manipulation Data in Robotic Strawberry Farming

TL;DR

The paper presents the Strawberry Robotic Operation Interface (SROI), an open-source handheld platform designed to collect dexterous manipulation demonstrations for strawberry farming in field environments. It combines a modular cut-and-hold end effector, a low-cost electric gripper, and stereo cameras with a portable data-collection unit, enabling accurate trajectory and gripper-state recording. A stereo-IMU SLAM-based post-processing pipeline (using ORB-SLAM3) recovers spatial poses without pre-mapped scenes, and an AprilTag-based detector provides gripper state, facilitating data-ready trajectories for imitation-learning pipelines. The authors validate the system with field experiments across two camera setups, achieving consistent, high-precision trajectory estimates (low RMSE in translation) and releasing an open strawberry-picking dataset to accelerate research in dexterous manipulation in agriculture. Overall, SROI offers a scalable, field-friendly data collection solution that supports data-driven approaches to automate occluded strawberry picking and related tasks.

Abstract

The strawberry farming is labor-intensive, particularly in tasks requiring dexterous manipulation such as picking occluded strawberries. To address this challenge, we present the Strawberry Robotic Operation Interface (SROI), an open-source device designed for collecting dexterous manipulation data in robotic strawberry farming. The SROI features a handheld unit with a modular end effector, a stereo robotic camera, enabling the easy collection of demonstration data in field environments. A data post-processing pipeline is introduced to extract spatial trajectories and gripper states from the collected data. Additionally, we release an open-source dataset of strawberry picking demonstrations to facilitate research in dexterous robotic manipulation. The SROI represents a step toward automating complex strawberry farming tasks, reducing reliance on manual labor.

Paper Structure

This paper contains 9 sections, 2 equations, 10 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Materials and Methods
  3. Hardware
  4. Software
  5. Experiments and Results
  6. Camera setups
  7. Workflow
  8. Trajectory estimation accuracy evaluation
  9. Conclusion

Figures (10)

  • Figure 1: Strawberries that are occluded by stems, leaves, and other strawberries necessitate the use of dexterous robotic manipulation for picking, as they cannot be directly harvested.
  • Figure 2: Left: Human demonstration of dexterous occluded strawberry picking using our handheld data collection device. Right: Robot arm strawberry picking using our e-gripper with same apperance as the handheld data collection device
  • Figure 3: The handheld data collection device of SROI including gripper, finger slider, camera mount and hand holder
  • Figure 4: Left: E-gripper has two linear electric actuators with same appearance and camera mount as the handheld device
  • Figure 5: Data collection unit with E-ink screen, USB port for connecting to camera and a user control button
  • ...and 5 more figures