Revealing Hierarchical Structure of Leaf Venations in Plant Science via Label-Efficient Segmentation: Dataset and Method

TL;DR

This work introduces HALVS, the first public dataset for hierarchical leaf vein segmentation with 1°, 2°, and 3° vein annotations across soybean, sweet cherry, and London planetree, captured using transmission-scanning and organized into 256×256 patches. It also proposes a label-efficient learning paradigm, Partially Supervised Semantic Segmentation (PSSS), that leverages fully labeled, partially labeled (1°/2° only), and unlabeled data within a semi-supervised framework to improve segmentation of the challenging 3° veins. Extensive experiments show that PSSS consistently boosts 3°-vein IoU across multiple SSL baselines and backbones, and ablations reveal the importance of the partial-label loss components and hyperparameters; cross-species experiments reveal transfer challenges due to domain shift. Overall, HALVS provides a valuable dataset and a practical methodology that advance plant phenomics by enabling hierarchical vein analysis with reduced labeling effort and by highlighting cross-species learning considerations.

Abstract

Hierarchical leaf vein segmentation is a crucial but under-explored task in agricultural sciences, where analysis of the hierarchical structure of plant leaf venation can contribute to plant breeding. While current segmentation techniques rely on data-driven models, there is no publicly available dataset specifically designed for hierarchical leaf vein segmentation. To address this gap, we introduce the HierArchical Leaf Vein Segmentation (HALVS) dataset, the first public hierarchical leaf vein segmentation dataset. HALVS comprises 5,057 real-scanned high-resolution leaf images collected from three plant species: soybean, sweet cherry, and London planetree. It also includes human-annotated ground truth for three orders of leaf veins, with a total labeling effort of 83.8 person-days. Based on HALVS, we further develop a label-efficient learning paradigm that leverages partial label information, i.e. missing annotations for tertiary veins. Empirical studies are performed on HALVS, revealing new observations, challenges, and research directions on leaf vein segmentation.

Figures (10)

• Figure 1: Illustration of three orders of veins (1°, 2°, and 3°) for three species of interest in HALVS. 1° veins, typically the thickest veins in the leaf, appear as the mid-vein of the leaf. 2° veins are the veins of the next smaller size that branch off from the 1° veins. 3° veins are the subsequent finer branches that stem from the 2° veins. Best viewed with digital zoom.
• Figure 2: Visualizations of leaves and corresponding vein annotations in HALVS (red: 1°, yellow: 2°, white 3°). Leaf patches are cropped in size of $256\times256$ for illustration purposes.
• Figure 3: Illustration of the proposed label-efficient learning framework for hierarchical leaf vein segmentation. The input of this framework includes unlabeled, partially labeled (i.e. 1° and 2° vein), and fully labeled (i.e. background, 1°, 2°, and 3° veins) data which are partitioned into leaf patches with a size of 256 × 256 pixels. Intuitively, complete leaf images instead of patches are used for illustration purposes. The labeled and unlabeled data are handled by a semi-supervised semantic segmentation method. The proposed partially supervised semantic segmentation (PSSS) module can be integrated with any semi-supervised learning framework to handle the partially labeled data. Here, a teacher-student model wang2022semi is depicted as an example (blue region). In PSSS (orange region), the teacher model generates pseudo-labels from the weakly augmented data, while the student model generates set of predictions $\mathcal{S}$ from the strongly augmented data. $\mathcal{S}_1$ contains the pixels that can be directly supervised by the ground truth of 1° and 2° veins from the partial labels. $\mathcal{S}_2$ contains the pixels that are predicted as pseudo-labels of background and 3° vein with high confidence. All remaining pixels are considered as 3° vein. Three sets of pixels are trained with three different losses.
• Figure 4: Visualizations of the segmentation results on the HALVS dataset. From top to bottom: soybean, sweet cherry, and London planetree. The colors of red, yellow, and white represent 1°, 2°, and 3° veins, respectively. The qualitative performance of the 3° vein in (e) is notably superior to the counterparts in (c) and (d). Best viewed with digital zoom.
• Figure 5: Comparison of performance under various combinations of score threshold $\tau$ and loss weight $\lambda$. The blue bar represents the optimal situation, with $\lambda = 1$ and $\tau = 0.95$.