Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploiting Boundary Loss for the Hierarchical Panoptic Segmentation of Plants and Leaves

Madeleine Darbyshire, Elizabeth Sklar, Simon Parsons

TL;DR

This work tackles the challenge of precision agriculture by developing a hierarchical panoptic segmentation framework that jointly identifies crops, weeds, and leaf instances while enabling leaf counting as a growth metric. It adapts Mask2Former with dual decoders (plants and leaves) and introduces focal loss and boundary loss, plus point sampling and deep supervision, to improve segmentation of small regions without increasing inference cost. On the PhenoBench UAV dataset, the approach achieves competitive panoptic quality (PQ) scores, with PQ$^{\dagger}$ reaching up to 83.70 using a SwinL backbone with test-time augmentation, and also yields improvements in leaf-count accuracy. The combination of lightweight architecture refinements and targeted loss functions demonstrates practical potential for real-world, data-efficient crop monitoring and targeted interventions in precision agriculture, with code available for reproducibility.

Abstract

Precision agriculture leverages data and machine learning so that farmers can monitor their crops and target interventions precisely. This enables the precision application of herbicide only to weeds, or the precision application of fertilizer only to undernourished crops, rather than to the entire field. The approach promises to maximize yields while minimizing resource use and harm to the surrounding environment. To this end, we propose a hierarchical panoptic segmentation method that simultaneously determines leaf count (as an identifier of plant growth)and locates weeds within an image. In particular, our approach aims to improve the segmentation of smaller instances like the leaves and weeds by incorporating focal loss and boundary loss. Not only does this result in competitive performance, achieving a PQ+ of 81.89 on the standard training set, but we also demonstrate we can improve leaf-counting accuracy with our method. The code is available at https://github.com/madeleinedarbyshire/HierarchicalMask2Former.

Exploiting Boundary Loss for the Hierarchical Panoptic Segmentation of Plants and Leaves

TL;DR

This work tackles the challenge of precision agriculture by developing a hierarchical panoptic segmentation framework that jointly identifies crops, weeds, and leaf instances while enabling leaf counting as a growth metric. It adapts Mask2Former with dual decoders (plants and leaves) and introduces focal loss and boundary loss, plus point sampling and deep supervision, to improve segmentation of small regions without increasing inference cost. On the PhenoBench UAV dataset, the approach achieves competitive panoptic quality (PQ) scores, with PQ reaching up to 83.70 using a SwinL backbone with test-time augmentation, and also yields improvements in leaf-count accuracy. The combination of lightweight architecture refinements and targeted loss functions demonstrates practical potential for real-world, data-efficient crop monitoring and targeted interventions in precision agriculture, with code available for reproducibility.

Abstract

Precision agriculture leverages data and machine learning so that farmers can monitor their crops and target interventions precisely. This enables the precision application of herbicide only to weeds, or the precision application of fertilizer only to undernourished crops, rather than to the entire field. The approach promises to maximize yields while minimizing resource use and harm to the surrounding environment. To this end, we propose a hierarchical panoptic segmentation method that simultaneously determines leaf count (as an identifier of plant growth)and locates weeds within an image. In particular, our approach aims to improve the segmentation of smaller instances like the leaves and weeds by incorporating focal loss and boundary loss. Not only does this result in competitive performance, achieving a PQ+ of 81.89 on the standard training set, but we also demonstrate we can improve leaf-counting accuracy with our method. The code is available at https://github.com/madeleinedarbyshire/HierarchicalMask2Former.
Paper Structure (27 sections, 8 equations, 4 figures, 4 tables)

This paper contains 27 sections, 8 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Semantic Segmentation
  4. Instance Segmentation
  5. Segmentation of Crops and Leaves
  6. Segmentation of Small Areas
  7. Methodology
  8. Adapted Mask2Former Architecture
  9. Loss
  10. Point Sampling
  11. Focal Loss
  12. Boundary Loss
  13. Loss Function
  14. Metrics
  15. Standard Panoptic Segmentation Metrics
  16. ...and 12 more sections

Figures (4)

  • Figure 1: Example images from the PhenoBench dataset. On the top are the full size images and the bottom shows a close-up of the same image. Images (a) are the originals, (b) show whole plants segmented at the pixel level, and (c) show individual leaves segmented at the pixel level.
  • Figure 2: Adapted Mask2Former architecture with an additional transformer decoder for segmenting leaves.
  • Figure 3: The boundary of ground truth area $G$, $\partial G$, is shown in green and the boundary of segmentation mask $S$, $\partial S$ is shown in orange.
  • Figure 4: Comparison of ground truth segmentations (top) with the segmentation results on SwinL ($L_f+L_b$) + TTA (bottom). The grey square in \ref{['fig:results:leaf']} demarcates the zoomed area shown in \ref{['fig:results:zoomed']}.