Raspberry PhenoSet: A Phenology-based Dataset for Automated Growth Detection and Yield Estimation

TL;DR

Raspberry PhenoSet is the first fruit dataset to integrate biology-based classification with fruit detection tasks, offering valuable insights for yield estimation and precise harvest timing, and its potential for both deep learning model development and practical robotic applications in agriculture is highlighted.

Abstract

The future of the agriculture industry is intertwined with automation. Accurate fruit detection, yield estimation, and harvest time estimation are crucial for optimizing agricultural practices. These tasks can be carried out by robots to reduce labour costs and improve the efficiency of the process. To do so, deep learning models should be trained to perform knowledge-based tasks, which outlines the importance of contributing valuable data to the literature. In this paper, we introduce Raspberry PhenoSet, a phenology-based dataset designed for detecting and segmenting raspberry fruit across seven developmental stages. To the best of our knowledge, Raspberry PhenoSet is the first fruit dataset to integrate biology-based classification with fruit detection tasks, offering valuable insights for yield estimation and precise harvest timing. This dataset contains 1,853 high-resolution images, the highest quality in the literature, captured under controlled artificial lighting in a vertical farm. The dataset has a total of 6,907 instances of mask annotations, manually labelled to reflect the seven phenology stages. We have also benchmarked Raspberry PhenoSet using several state-of-the-art deep learning models, including YOLOv8, YOLOv10, RT-DETR, and Mask R-CNN, to provide a comprehensive evaluation of their performance on the dataset. Our results highlight the challenges of distinguishing subtle phenology stages and underscore the potential of Raspberry PhenoSet for both deep learning model development and practical robotic applications in agriculture, particularly in yield prediction and supply chain management. The dataset and the trained models are publicly available for future studies.

Figures (6)

• Figure 1: A sample image from the Raspberry PhenoSet, taken at the vertical farming facility at the Center of Urban Innovation of Toronto Metropolitan University. a) Original image with no annotations. b) Mask annotations of all instances present in the image; Labels A-G correspond to the seven phenology stages of raspberries.
• Figure 2: Phenology reference of raspberry development stages used for annotating the images. The stages are labelled a-g, representing the seven development stages. a) Buds b) Open Flower c) Fruit Initiation d) Green Fruit e) Growing Fruit (yellow colour) f) Semi-mature (pink) fruit g) Mature (red) fruit
• Figure 3: Histogram of Annotations in Raspberry PhenoSet a) Number of Annotations vs. Pixel Heights b) Number of Annotations vs. Pixel Widths c) Number of Images vs. Number of Annotations per Image
• Figure 4: Best Performance of Different Networks on the Raspberry PhenoSet, Visualizing the Suitability of YOLOv8 and RT-DETR for Fruit Detection Applications
• Figure 5: Performance of YOLOv8-x on Each Class of the Raspberry PhenoSet, Showing the Model's Ability to Distinguish Each Phenology Stage