Comparing YOLOv11 and YOLOv8 for instance segmentation of occluded and non-occluded immature green fruits in complex orchard environment

TL;DR

This work systematically compares YOLOv11 and YOLOv8 across multiple instance segmentation configurations for occluded and non-occluded immature green fruits in a real orchard. Using a rigorously controlled dataset and uniform hyperparameters, it demonstrates that YOLO11m-seg delivers the strongest overall and non-occluded segmentation performance (mAP@50 in the 0.908–0.909 range) while YOLOv8n offers the fastest inference (3.3 ms). The study also inventories computational demands (parameters, GFLOPs, layers) and training dynamics (epochs, hours), highlighting tradeoffs between accuracy and speed. The results provide actionable guidance for deploying real-time, high-precision fruit segmentation in automated orchard systems, with future work pointing to newer YOLO versions, NAS, and transfer learning to further boost robustness and efficiency in agricultural imaging.

Abstract

This study conducted a comprehensive performance evaluation on YOLO11 (or YOLOv11) and YOLOv8, the latest in the "You Only Look Once" (YOLO) series, focusing on their instance segmentation capabilities for immature green apples in orchard environments. YOLO11n-seg achieved the highest mask precision across all categories with a notable score of 0.831, highlighting its effectiveness in fruit detection. YOLO11m-seg and YOLO11l-seg excelled in non-occluded and occluded fruitlet segmentation with scores of 0.851 and 0.829, respectively. Additionally, YOLOv11x-seg led in mask recall for all categories, achieving a score of 0.815, with YOLO11m-seg performing best for non-occluded immature green fruitlets at 0.858 and YOLOv8x-seg leading the occluded category with 0.800. In terms of mean average precision at a 50\% intersection over union (mAP@50), YOLOv11m-seg consistently outperformed, registering the highest scores for both box and mask segmentation, at 0.876 and 0.860 for the "All" class and 0.908 and 0.909 for non-occluded immature fruitlets, respectively. YOLO11l-seg and YOLOv8l-seg shared the top box mAP@50 for occluded immature fruitlets at 0.847, while YOLO11m-seg achieved the highest mask mAP@50 of 0.810. Despite the advancements in YOLO11, YOLOv8n surpassed its counterparts in image processing speed, with an impressive inference speed of 3.3 milliseconds, compared to the fastest YOLO11 series model at 4.8 milliseconds, underscoring its suitability for real-time agricultural applications related to complex green fruit environments. (YOLOv11 segmentation)

Figures (10)

• Figure 1: Architecture diagram of YOLOv8 algorithm: YOLOv8 advances real-time object detection with its innovative backbone and anchor-free Ultralytics head, optimizing detection accuracy and speed across various tasks (primary image source: https://yolov8.org/what-is-yolov8/).
• Figure 2: Architecture diagram of YOLO11 algorithm: YOLO11 builds on YOLOv8's architecture with refined detection and segmentation efficiency and accuracy over benchmark dataset sapkota2024yolov10
• Figure 3: YOLO11 and YOLOv8 performance results on benchmark datasets, showcasing YOLO11's superior accuracy and latency against predecessors and competitors. YOLO11 achieves the highest mAP scores at varied latency levels, highlighting its efficiency in real-time applications:(a) Performance comparison of YOLO models on the COCO dataset, ; (b) Performance comparison between YOLO11 and YOLOv8 on the COCO dataset
• Figure 4: Data acquisition, annotation and training deep learning process diagram: (a) Robotic imaging platform equipped with a Microsoft Azure Kinect DK sensor on a UR5e manipulator arm, navigating through an orchard; (b) Example of RGB images captured showing complex backgrounds; (c) Manual annotation of occluded and non-occluded immature green fruits; (d) Process diagram: prepared dataset split into training, testing, and validation sets.
• Figure 5: Examples showing YOLO11n and YOLOv8n's results in segmenting immature green fruits in a commercial Scilate orchard. The top row displays original RGB images, the middle row illustrates YOLO11n's segmentation results, and the bottom row showcases YOLOv8n's performance on the same images.